Image processing apparatus and method

ABSTRACT

An image processing apparatus (encoder, decoder or the like) and method suppress reduction in encoding efficiency. Encoding of coefficient data is skipped in an invalid transform coefficient region, and the coefficient data is encoded in a valid transform coefficient region. Further, for example, the coefficient data in the valid transform coefficient region is encoded in a scan order corresponding to a block shape of a block to be processed. In addition, decoding of encoded data including encoded coefficient data related to an image is skipped in an invalid transform coefficient region, and the encoded data is decoded in a valid transform coefficient region. Further, for example, the encoded data in the valid transform coefficient region is decoded in a scan order corresponding to the block shape of the block to be processed.

TECHNICAL FIELD

The present disclosure relates to image processing apparatus and method,and particularly, to image processing apparatus and method that cansuppress a reduction in encoding efficiency.

BACKGROUND ART

In the past, a process (Zero Out) of setting high-frequency regioncoefficients of a block in a size larger than 32×32 to 0 has beenproposed in encoding and decoding of image data. For example, in a casewhere the block size is equal to or greater than 64, the abovementionedprocess is executed to transform and encode the image data of the blockinto valid coefficient data in a region of 32×32 on the upper left ofthe block and into coefficient data with a value of “0” in other regions(for example, see NPL 1).

CITATION LIST Non Patent Literature [NPL 1]

-   Benjamin Bross, Jianle Chen, Shan Liu, “Versatile Video Coding    (Draft 2),” JVET-K1001-v7, Joint Video Experts Team (JVET) of ITU-T    SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 11th Meeting: Ljubljana,    SI, 10-18 Jul. 2018

SUMMARY Technical Problems

However, when the abovementioned process is executed to perform encodingin an existing order of scan, there is a risk of encoding coefficientdata in an invalid transform coefficient region not including validcoefficient data. Therefore, the encoding efficiency may be reduced byunnecessary information.

The present disclosure has been made in view of the circumstances, andthe present disclosure can suppress a reduction in encoding efficiency.

Solution to Problems

An aspect of the present technique provides an image processingapparatus including an encoding unit that skips encoding of coefficientdata related to an image in an invalid transform coefficient region andthat encodes the coefficient data in a valid transform coefficientregion.

The aspect of the present technique provides an image processing methodincluding skipping encoding of coefficient data related to an image inan invalid transform coefficient region and encoding the coefficientdata in a valid transform coefficient region.

Another aspect of the present technique provides an image processingapparatus including a decoding unit that skips, in an invalid transformcoefficient region, decoding of encoded data including encodedcoefficient data related to an image and that decodes the encoded datain a valid transform coefficient region.

The other aspect of the present invention provides an image processingmethod including skipping decoding of encoded data including encodedcoefficient data related to an image in an invalid transform coefficientregion and decoding the encoded data in a valid transform coefficientregion.

In the image processing apparatus and method according to the aspect ofthe present technique, the encoding of the coefficient data related tothe image is skipped in the invalid transform coefficient region, andthe coefficient data is encoded in the valid transform coefficientregion.

In the image processing apparatus and method according to the otheraspect of the present technique, the decoding of the encoded dataincluding the encoded coefficient data related to the image is skippedin the invalid transform coefficient region, and the encoded data isdecoded in the valid transform coefficient region.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 illustrates diagrams describing an example of a block structure.

FIG. 2 is another diagram describing an example of the block structure.

FIG. 3 is a diagram describing an example of Zero Out.

FIG. 4 is a diagram describing an example of an order of scan.

FIG. 5 is another diagram describing an example of the order of scan.

FIG. 6 is a diagram describing an example of syntax.

FIG. 7 is a diagram describing an example of encoding.

FIG. 8 is another diagram describing an example of the encoding.

FIG. 9 is still another diagram describing an example of the encoding.

FIG. 10 is yet another diagram describing an example of the encoding.

FIG. 11 is a further diagram describing an example of the encoding.

FIG. 12 is a still further diagram describing an example of theencoding.

FIG. 13 is a diagram describing an example of scan control.

FIG. 14 is a diagram describing an example of skipping an invalidtransform coefficient region.

FIG. 15 is another diagram describing an example of skipping the invalidtransform coefficient region.

FIG. 16 is still another diagram describing an example of skipping theinvalid transform coefficient region.

FIG. 17 is a block diagram illustrating a main configuration example ofan encoding apparatus.

FIG. 18 is a flow chart describing an example of a flow of an encodingprocess.

FIG. 19 is a block diagram illustrating a main configuration example ofa decoding apparatus.

FIG. 20 is a flow chart describing an example of a flow of a decodingprocess.

FIG. 21 is another diagram describing an example of the syntax.

FIG. 22 is a diagram describing an example of scan order control of avalid transform coefficient region.

FIG. 23 is another diagram describing an example of the scan ordercontrol of the valid transform coefficient region.

FIG. 24 is still another diagram describing an example of the scan ordercontrol of the valid transform coefficient region.

FIG. 25 is yet another diagram describing an example of the scan ordercontrol of the valid transform coefficient region.

FIG. 26 is a further diagram describing an example of the scan ordercontrol of the valid transform coefficient region.

FIG. 27 is a further diagram describing an example of the scan ordercontrol of the valid transform coefficient region.

FIG. 28 is a further diagram describing an example of the scan ordercontrol of the valid transform coefficient region.

FIG. 29 is a further diagram describing an example of the scan ordercontrol of the valid transform coefficient region.

FIG. 30 is a further diagram describing an example of the scan ordercontrol of the valid transform coefficient region.

FIG. 31 is a further diagram describing an example of the scan ordercontrol of the valid transform coefficient region.

FIG. 32 is a further diagram describing an example of the scan ordercontrol of the valid transform coefficient region.

FIG. 33 illustrates diagrams describing an example of a scan ordercalculation method.

FIG. 34 is another diagram describing an example of the scan ordercalculation method.

FIG. 35 is a further diagram describing an example of the scan ordercontrol of the valid transform coefficient region.

FIG. 36 illustrates other diagrams describing an example of the scanorder calculation method.

FIG. 37 is still another diagram describing an example of the scan ordercalculation method.

FIG. 38 is a further diagram describing an example of the scan ordercontrol of the valid transform coefficient region.

FIG. 39 is a further diagram describing an example of the scan ordercontrol of the valid transform coefficient region.

FIG. 40 is a further diagram describing an example of the scan ordercontrol of the valid transform coefficient region.

FIG. 41 is a further diagram describing an example of the scan ordercontrol of the valid transform coefficient region.

FIG. 42 is a diagram describing an example of skipping the invalidtransform coefficient region of chroma components.

FIG. 43 is a block diagram illustrating a main configuration example ofan image encoding apparatus.

FIG. 44 is a flow chart illustrating an example of a flow of an imageencoding process.

FIG. 45 is a block diagram illustrating a main configuration example ofan image decoding apparatus.

FIG. 46 is a flow chart describing an example of a flow of an imagedecoding process.

FIG. 47 is still another diagram describing an example of the syntax.

FIG. 48 is yet another diagram describing an example of the syntax.

FIG. 49 is a block diagram illustrating a main configuration example ofa computer.

DESCRIPTION OF EMBODIMENTS

Hereinafter, modes for carrying out the present disclosure (hereinafter,referred to as embodiments) will be described. Note that the embodimentswill be described in the following order.

1. Literature and the Like Supporting Technical Content and TechnicalTerms

2. Zero Out

3. Concept

4. First Embodiment (Method 1)

5. Second Embodiment (Method 1-1)

6. Third Embodiment (Method 1-2)

7. Fourth Embodiment (Image Encoding Apparatus and Image DecodingApparatus)

8. Note

1. Literature and the Like Supporting Technical Content and TechnicalTerms

The scope disclosed in the present technique includes not only thecontent described in the embodiments, but also the content described inthe following pieces of Non Patent Literature publicly known at the timeof the application.

-   NPL 1: (described above)-   NPL 2: TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU    (International Telecommunication Union), “Advanced video coding for    generic audiovisual services,” H.264, April 2017-   NPL 3: TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU    (International Telecommunication Union), “High efficiency video    coding,” H.265, December 2016-   NPL 4: Jianle Chen, Elena Alshina, Gary J. Sullivan, Jens-Rainer,    Jill Boyce, “Algorithm Description of Joint Exploration Test Model    4,” JVET-G1001_v1, Joint Video Exploration Team (JVET) of ITU-T SG    16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 7th Meeting: Torino, IT, 13-21    Jul. 2017

That is, the content described in the abovementioned pieces of NonPatent Literature is also a basis for determining the supportrequirements. For example, even in a case where the Quad-Tree BlockStructure described in NPL 3 or the QTBT (Quad Tree Plus Binary Tree)Block Structure described in NPL 4 is not directly described in theembodiments, they are within the disclosed scope of the presenttechnique, and the support requirements of the claims are satisfied.Further, even in a case where, for example, technical terms, such asparse (Parsing), syntax (Syntax), and semantics (Semantics), are notdirectly described in the embodiments, they are similarly within thedisclosed scope of the present technique, and the support requirementsof the claims are satisfied.

Further, in the present specification, a “block” (not a block indicatinga processing unit) used for describing a partial region or a unit ofprocessing of an image (picture) indicates any partial region in thepicture unless otherwise stated, and the dimension, the shape, thecharacteristics, and the like of the “block” are not limited. Forexample, the “block” includes any partial region (unit of processing),such as TB (Transform Block), TU (Transform Unit), PB (PredictionBlock), PU (Prediction Unit), SCU (Smallest Coding Unit), CU (CodingUnit), LCU (Largest Coding Unit), CTB (Coding Tree Block), CTU (CodingTree Unit), transform block, sub-block, macroblock, tile, and slicedescribed in NPL 2 to NPL 4.

Further, in designating the size of the block, the block size may notonly directly be designated, but may also indirectly be designated. Forexample, identification information for identifying the size may be usedto designate the block size. In addition, for example, the ratio ordifference from the size of a reference block (for example, LCU, SCU, orthe like) may be used to designate the block size. For example, in acase where information for designating the block size is transmitted ina syntax element or the like, the information for indirectly designatingthe size may be used as such information. In such a way, the amount ofinformation can be reduced, and the encoding efficiency may be improved.In addition, the designation of the block size also includes designationof a range of the block size (for example, designation of allowed rangeof block size or the like).

In addition, encoding in the present specification includes not only theentire process of transforming an image into a bitstream, but also partof the process. For example, the encoding includes not only a processincluding a prediction process, an orthogonal transform, quantization,arithmetic coding, and the like, but also a process representing thequantization and the arithmetic coding as a whole, a process includingthe prediction process, the quantization, and the arithmetic coding, andthe like. Similarly, decoding includes not only the entire process oftransforming a bitstream into an image, but also part of the process.For example, the decoding includes not only a process including inversearithmetic decoding, inverse quantization, an inverse orthogonaltransform, a prediction process, and the like, but also a processincluding the inverse arithmetic decoding and the inverse quantization,a process including the inverse arithmetic decoding, the inversequantization, and the prediction process, and the like.

2. Zero Out <Block Structure of Encoding and Decoding>

In the test model (JEM4 (Joint Exploration Test Model 4)) described inNPL 4, the concept of CTUs (coding tree units) is adopted as in HEVC(High Efficiency Video Coding) described in NPL 3. That is, in the caseof the method, the picture (Picture) is divided into CTUs, and the CTUsare set as the highest block units in encoding. In NPL 4, the maximumsize of the CTU is defined as 128×128. However, the maximum size of Lumatransform block (TB) is 64×64.

In HEVC, a quad-tree (quadtree) structure is used to further divide theCTU into CUs (coding units). A PU (prediction unit) and a TU (TransformUnit) are provided in the CU, and the PU size and the TU size canindependently be selected according to the features of the input image.The quaternary-tree structure can be used to divide the TU, similarly tothe CU.

On the other hand, in the test model described in NPL 4, a multi-typetree (binary-tree, ternary-tree)+quad-tree structure is adopted in placeof the concept of CU/PU/TU in HEVC. That is, as illustrated in A of FIG.1, a block 11 can be divided into four pieces based on the quad-tree asin a block 11-1, can be divided into two pieces based on the binary-treeas in a block 11-2 and a block 11-3, or can be divided into three piecesbased on the ternary-tree as in a block 11-4 and a block 11-5.Therefore, a CTU 12 can be divided into various blocks as illustrated inB of FIG. 1.

FIG. 2 illustrates an example of the multi-type tree. In the case of thestructure, the CTU is first divided based on the quad-tree. Thequad-tree leaf is further divided into multi-type tree structures.Further, more flexible block division can also be performed based on thebinary-tree structure or the ternary-tree structure. Therefore, as inthe example of B in FIG. 1, the CTU 12 can also be divided intorectangular CUs.

The multi-type tree leaf corresponds to the CU size. The size of CU isthe same as the size of PU and the size of TU unless the size of CUexceeds the maximum size of the Transform size. In a case where the CUsize exceeds the Transform size, the TU is automatically divided up tothe upper limit. The block size in this case is CU=PU !=TU.

<Large Block Size Transform>

While the block size of the orthogonal transform of HEVC is 32×32 atmost, the block size of the orthogonal transform is extended to 64×64 atmost in the case of the test model described in NPL 4. In NPL 1,proposed is a process (Zero Out), in which a high-frequency regioncoefficient of a block in a size larger than 32×32 is set to 0. Forexample, in a case where the block size is equal to or greater than 64like a block 20 illustrated in FIG. 3, the abovementioned process isexecuted to transform and encode the image data into valid coefficientdata (also referred to as a valid transform coefficient) in a region of32×32 on the upper left of the block 20 and into coefficient data with avalue of “0” in other regions. The region including the valid transformcoefficient will also be referred to as a valid transform coefficientregion. That is, a valid transform coefficient region 21 of the block 20in FIG. 3 is a region including valid transform coefficients.

In addition, a region other than the valid transform coefficient regionin the block will also be referred to as an invalid transformcoefficient region. The invalid transform coefficient region (in thecase of the example in FIG. 3, invalid transform coefficient region 22)is a region including coefficient data that is not valid (also referredto as invalid transform coefficients). The invalid transform coefficientis, for example, coefficient data with a value of “0.”

The pieces of coefficient data (transform coefficients) of the block 20are encoded in the order of scan (scan order) as in an exampleillustrated in FIG. 4. The pieces of coefficient data are processed onthe basis of 4×4 sub-blocks, and the sub-blocks are processed in orderfrom sub-blocks of components in higher ranges to sub-blocks ofcomponents in lower ranges. In the case of the example in FIG. 4,sub-blocks 23 are formed in the block, and the sub-blocks 23 are scannedin the order indicated by arrows from the sub-block on the lower rightedge in FIG. 4 to the sub-block on the upper left edge in FIG. 4(diagonal order from lower left to upper right is repeated from lowerright to upper left). Note that scan in the reverse order of the arrowsin the example is also possible (diagonal order from upper right tolower left is repeated from lower right to upper left).

In each sub-block 23, valid transform coefficient flag information(coded_sub_block_flag) included in the coefficient data and indicatingthe presence/absence of the valid transform coefficient for eachsub-block is encoded by context-based binary arithmetic coding (contextadaptive binary arithmetic coding). Further, validity flag information(sig_coeff_flag), which is included in the coefficient data and whichindicates, for each sub-block, the presence/absence of the validtransform coefficient with a value that is not 0, is encoded. Further,for the sub-block in which the validity flag information(sig_coeff_flag) is true (value indicating that the sub-block includes avalid transform coefficient with a value that is not 0, such as a valueof “1”), a sign (coeff_sign_flag) and an absolute value (abs) of eachvalid transform coefficient are encoded.

However, in a case where last_pos indicating the position of thesub-block including valid coefficient data in the highest range of theblock is set, the scan is started from the sub-block at the positionindicated by last_pos. For example, in a case where last_pos indicatesthe position of a sub-block 41 including valid coefficient data in thehighest range of the block 20 as in an example of FIG. 5, the scan isstarted from the sub-block 41 based on the information. That is, of thescan in the block illustrated in FIG. 4, scan indicated by arrows ofsolid lines in the block 20 illustrated in FIG. 5 (that is, scan from asub-block 41 to a sub-block on the upper left edge of the block 20) isexecuted. In other words, the scan indicated by arrows of dotted linesin FIG. 5 (that is, scan from the sub-block on the lower right edge ofthe block 20 to the sub-block 41) is skipped.

In this case, there is a case in which the range of the scan includes aninvalid transform coefficient region as in the example of FIG. 5 (graysub-blocks in FIG. 5). In that case, the sub-blocks in the invalidtransform coefficient region are also scanned. Note that the invalidtransform coefficient region does not include a valid transformcoefficient, and therefore, the valid transform coefficient flaginformation (codec sub block flag) of the sub-block in the invalidtransform coefficient region is false (value indicating that thesub-block does not include a valid transform coefficient, such as avalue of “0”).

<Syntax>

FIG. 6 illustrates an example of syntax. In the example, processing isrepeated from the sub-block (lastSubBlock) including the validcoefficient data of the highest range to the sub-block at the top of theblock (sub-block on the upper left edge) in the for statement in thefirst stage from the top. In addition, the position in the x-axisdirection and the y-axis direction is updated in the third stage and thefourth stage from the top, and the valid transform coefficient flaginformation (coded_sub_block_flag) is set for the sub-block at eachposition in the third row from the bottom. That is, the coefficient data(for example, valid transform coefficient flag information(coded_sub_block_flag)) is encoded for each sub-block from the sub-block(lastSubBlock) including the valid coefficient data in the highest rangeto the sub-block (sub-block on the upper left edge) at the top of theblock.

<Encoding of Invalid Transform Coefficient Region>

However, in the case of the scan method as described above, thecoefficient data in the invalid transform coefficient region is alsoscanned and encoded as illustrated in FIG. 5. However, it is apparentthat the coefficient data in the invalid transform coefficient regionhas a value of “0,” and the transmission is not necessary. That is, thecoefficient data in the invalid transform coefficient region can bederived on the decoding side without the transmission from the encodingside.

That is, the encoding efficiency may be reduced by the unnecessaryinformation in the scan method described above. In addition, theencoding and the decoding of the unnecessary information may alsoincrease the load of the encoding process and the decoding process.

For example, in a case where the scan is started from the sub-block 41at the lower right corner in the 32×32 valid transform coefficientregion (white part in FIG. 7) on the upper left in the 64×64 block 20 asin an example of FIG. 7, dark gray sub-blocks in the invalid transformcoefficient region (gray sub-blocks) are scanned.

In addition, this is similar in a case where, for example, the scan isstarted from the sub-block 41 (in the valid transform coefficientregion) at the position of 5×5 from the upper left corner of the 64×64block 20 as in an example of FIG. 8, and dark gray sub-blocks in theinvalid transform coefficient region (gray sub-blocks) are scanned.

This is similar in a case where the block 20 is horizontally long. Forexample, as in FIG. 9, it is assumed that 32×32 on the left is a validtransform coefficient region (white part in FIG. 9) and 32×32 on theright is an invalid transform coefficient region in the 64×32 block 20.In a case where the scan is started from the sub-block 41 at the lowerright corner of the valid transform coefficient region (white part inFIG. 9) in such a block 20, dark gray sub-blocks in the invalidtransform coefficient region (gray sub-blocks) are scanned.

This is similar in a case where the block 20 is vertically long. Forexample, as in FIG. 10, it is assumed that upper 32×32 is a validtransform coefficient region (white part in FIG. 10) and lower 32×32 isan invalid transform coefficient region in the 32×64 block 20. In a casewhere the scan is started from the sub-block 41 at the lower rightcorner of the valid transform coefficient region (white part in FIG. 10)in such a block 20, dark gray sub-blocks in the invalid transformcoefficient region (gray sub-blocks) are scanned.

This is similar in a case where the block 20 is horizontally long. Forexample, as in FIG. 11, it is assumed that 32×16 on the left is a validtransform coefficient region (white part in FIG. 11) and 32×16 on theright is an invalid transform coefficient region in the 64×16 block 20.In a case where the scan is started from the sub-block 41 at the lowerright corner of the valid transform coefficient region (white part inFIG. 11) in such a block 20, dark gray sub-blocks in the invalidtransform coefficient region (gray sub-blocks) are scanned.

This is similar in a case where the block 20 is vertically long. Forexample, as in FIG. 12, it is assumed that upper 16×32 is a validtransform coefficient region (white part in FIG. 12) and lower 16×32 isan invalid transform coefficient region in the 16×64 block 20. In a casewhere the scan is started from the sub-block 41 at the lower rightcorner of the valid transform coefficient region (white part in FIG. 12)in such a block 20, dark gray sub-blocks in the invalid transformcoefficient region (gray sub-blocks) are scanned.

In such a way, unnecessary information is encoded and decoded regardlessof the shape of the block in the scan method described above, and thismay lead to a reduction in the encoding efficiency, an increase in theload of the encoding process and the decoding process, or the like.

3. Concept <Method 1>

Therefore, as indicated in the first stage from the top of a table inFIG. 13, the abovementioned scan is controlled to skip the invalidtransform coefficient region.

For example, encoding of coefficient data related to an image is skippedin the invalid transform coefficient region, and the coefficient data isencoded in the valid transform coefficient region. In addition, forexample, the image processing apparatus includes an encoding unit thatskips encoding of coefficient data related to an image in the invalidtransform coefficient region and that encodes the coefficient data inthe valid transform coefficient region. In such a way, encoding anddecoding of the coefficient data in the invalid transform coefficientregion can be suppressed, that is, encoding and decoding of unnecessaryinformation can be suppressed. This can suppress the reduction in theencoding efficiency. In addition, for a similar reason, an increase inthe load of the encoding process can be suppressed, and an increase inthe cost, the circuit scale, the processing time, and the like can besuppressed (typically, the device can be more inexpensively developedand manufactured, the device can be more easily downsized, and theencoding can be performed faster).

In addition, for example, decoding of encoded data including encodedcoefficient data related to an image is skipped in the invalid transformcoefficient region, and the encoded data is decoded in the validtransform coefficient region. In addition, for example, the imageprocessing apparatus includes a decoding unit that skips decoding ofencoded data including encoded coefficient data related to an image inthe invalid transform coefficient region and that decodes the encodeddata in the valid transform coefficient region. In such a way, encodingand decoding of the coefficient data in the invalid transformcoefficient region can be suppressed, that is, encoding and decoding ofunnecessary information can be suppressed. This can suppress thereduction in the encoding efficiency (typically, the encoding efficiencycan be improved). In addition, for a similar reason, an increase in theload of the decoding process can be suppressed, and an increase in thecost, the circuit scale, the processing time, and the like can besuppressed (typically, the device can be more inexpensively developedand manufactured, the device can be more easily downsized, and thedecoding can be performed faster).

<Method 1-1>

In addition, although the sub-blocks are scanned in the diagonaldirection in the block in the description above, the scan direction ofthe sub-block units is optional, and the scan direction may be adirection other than the diagonal direction. For example, in addition tothe method 1 described above, the order of scan in the valid transformcoefficient region of the block to be processed may be controlled asindicated in the second stage from the top of the table in FIG. 13, forexample.

For example, the coefficient data of the valid transform coefficientregion may be encoded in a scan order corresponding to the tendency ofthe distribution (features of distribution) of the values of thecoefficient data in the block to be processed. In addition, for example,the encoded data of the valid transform coefficient region may bedecoded in a scan order corresponding to the tendency of thedistribution of the values of the coefficient data in the block to beprocessed. Although the details will be described later, thedistribution of the values of the coefficient data varies according tothe block shape, the prediction method, and the like. Therefore, theorder of scan can be controlled in such a way to encode and decode thecoefficient data by using the order of scan more suitable for thetendency of the distribution of the values of the coefficient data.Therefore, the reduction in the encoding efficiency can be suppressed(typically, encoding efficiency can be improved).

<Method 1-1-1>

For example, the scan direction in the valid transform coefficientregion may be vertical or horizontal as indicated in the third stagefrom the top of the table in FIG. 13. That is, for example, thecoefficient data of the valid transform coefficient region may bescanned and encoded in the vertical direction or the horizontaldirection for each sub-block. In addition, for example, the encoded dataof the valid transform coefficient region may be scanned and decoded inthe vertical direction or the horizontal direction for each sub-block.

<Method 1-1-2>

In addition, for example, the scan direction in the valid transformcoefficient region may be a direction corresponding to the block shapeas indicated in the fourth stage from the top of the table in FIG. 13.That is, for example, the coefficient data of the valid transformcoefficient region may be encoded for each sub-block in the scan ordercorresponding to the shape of the block to be processed. In addition,for example, the encoded data of the valid transform coefficient regionmay be decoded for each sub-block in the scan order corresponding to theshape of the block to be processed. Note that the scan direction of thesub-block units is not limited to the straight line direction (singledirection). For example, the scan direction may be a curve direction ormay include plural scan directions.

<Method 1-1-3>

In addition, for example, the sub-blocks of the valid transformcoefficient region may be grouped to control the scan as indicated inthe fifth stage from the top of the table in FIG. 13. That is, forexample, the coefficient data of the valid transform coefficient regionmay be scanned and encoded for each group of sub-blocks. In addition,for example, the encoded data of the valid transform coefficient regionmay be scanned and decoded for each group of sub-blocks.

<Method 1-2>

Further, in addition to the method 1 described above, the chromacomponents of the block to be processed may also control the scansimilarly to the luminance components, as indicated in the sixth stagefrom the top of the table in FIG. 13, for example. For example, for theluminance components and the chroma components of the image, encoding ofthe coefficient data may be skipped in the invalid transform coefficientregion, and the coefficient data may be encoded in the valid transformcoefficient region. In addition, for example, for the luminancecomponents and the chroma components of the image, decoding of theencoded data may be skipped in the invalid transform coefficient region,and the encoded data may be decoded in the valid transform coefficientregion.

In such a way, encoding and decoding of unnecessary information can besuppressed not only for the luminance components, but also for thechroma components. Therefore, the reduction in the encoding efficiencycan be suppressed (typically, the encoding efficiency can be improved).In addition, for a similar reason, the increase in the load of thedecoding process can be suppressed, and the increase in the cost, thecircuit scale, the processing time, and the like can be suppressed(typically, the device can be more inexpensively developed andmanufactured, the device can be more easily downsized, and decoding canbe performed faster).

4. First Embodiment <4-1: Scan Example of Method 1>

Next, each method in FIG. 13 will specifically be described. In thepresent embodiment, the “method 1” will be described. As describedabove, encoding and decoding are skipped in the invalid transformcoefficient region, and encoding and decoding are performed in the validtransform coefficient region in the method 1. That is, only thecoefficient data in the valid transform coefficient region is encodedand decoded as in an example of FIG. 14.

In the case of the example in FIG. 14, the scan is started from thesub-block 41 at the lower right corner of the 32×32 valid transformcoefficient region on the upper left (white part in FIG. 14) in the64×64 block 20 as in the example of FIG. 7. However, the control isperformed as described above in this case, and only the sub-blocks inthe valid transform coefficient region are encoded and decoded asindicated by arrows in the block 20 of FIG. 14. That is, the sub-blocksin the valid transform coefficient region are scanned in order from anarrow 101 to an arrow 115, and the sub-blocks in the invalid transformcoefficient region are not scanned.

This can suppress encoding and decoding of the sub-blocks in the invalidtransform coefficient region that are unnecessary information, and thereduction in the encoding efficiency can be suppressed. In addition, fora similar reason, the increase in the load of the encoding process andthe decoding process can be suppressed, and the increase in the cost,the circuit scale, the processing time, and the like can be suppressed(typically, the device can be more inexpensively developed andmanufactured, the device can be more easily downsized, and the encodingand the decoding can be performed faster).

The shape of the block 20 is optional, and the shape may not be a squareas in the example of FIG. 14. For example, the block 20 may be ahorizontally long rectangle as in an example of FIG. 15. In the case ofthe example of FIG. 15, the size of the block 20 is 64×32 as in the caseof FIG. 9. In addition, 32×32 on the left is a valid transformcoefficient region (white part in FIG. 15), and 32×32 on the right is aninvalid transform coefficient region. The scan is started from thesub-block 41 at the lower right corner of the valid transformcoefficient region (white part in FIG. 15). However, the control isperformed as described above in this case, and only the sub-blocks inthe valid transform coefficient region are encoded and decoded asindicated by arrows in the block 20 of FIG. 15. That is, the sub-blocksin the valid transform coefficient region are scanned in order from anarrow 121 to an arrow 135, and the sub-blocks in the invalid transformcoefficient region are not scanned.

This can suppress encoding and decoding of the sub-blocks in the invalidtransform coefficient region that are unnecessary information, and thereduction in the encoding efficiency can be suppressed. In addition, fora similar reason, the increase in the load of the encoding process andthe decoding process can be suppressed, and the increase in the cost,the circuit scale, the processing time, and the like can be suppressed(typically, the device can be more inexpensively developed andmanufactured, the device can be more easily downsized, and the encodingand the decoding can be performed faster).

In addition, for example, the block 20 may be a vertically longrectangle as in an example of FIG. 16. In the case of the example inFIG. 16, the size of the block 20 is 32×64 as in the case of FIG. 10. Inaddition, upper 32×32 is a valid transform coefficient region (whitepart in FIG. 16), and lower 32×32 is an invalid transform coefficientregion. The scan is started from the sub-block 41 at the lower rightcorner of the valid transform coefficient region (white part in FIG.16). However, the control is performed as described above in this case,and only the sub-blocks in the valid transform coefficient region areencoded and decoded as indicated by arrows in the block 20 of FIG. 16.That is, the sub-blocks in the valid transform coefficient region arescanned in order from an arrow 151 to an arrow 165, and the sub-blocksin the invalid transform coefficient region are not scanned.

This can suppress encoding and decoding of the sub-blocks in the invalidtransform coefficient region that are unnecessary information, and thereduction in the encoding efficiency can be suppressed. In addition, fora similar reason, the increase in the load of the encoding process andthe decoding process can be suppressed, and the increase in the cost,the circuit scale, the processing time, and the like can be suppressed(typically, the device can be more inexpensively developed andmanufactured, the device can be more easily downsized, and the encodingand the decoding can be performed faster).

Note that, although the diagonal order from lower left to upper right isrepeated from lower right to upper left in the example of the order ofscan (scan direction) described above as in the example of FIG. 4, theorder of scan in which the present technique is applied is optional, andthe order of scan is not limited to the example. For example, a diagonalorder from upper right to lower left may be repeated from lower right toupper left in the opposite direction from the arrows in FIG. 4.

<4-2: Encoding> <Encoding Apparatus>

Next, an example of the device that realizes such scan control will bedescribed. FIG. 17 is a block diagram illustrating an example of a mainconfiguration of an encoding apparatus as a mode of the image processingapparatus according to the present technique. An encoding apparatus 170illustrated in FIG. 17 is an apparatus that encodes coefficient data ofan input image and that outputs the encoded data. As illustrated in FIG.17, the encoding apparatus 170 includes a scan control unit 171 and anencoding unit 172.

The scan control unit 171 executes a process related to the control ofscan. For example, the scan control unit 171 acquires encodinginformation input from the outside of the encoding apparatus 170. Theencoding information is also called encoding parameters and includes anyinformation regarding encoding of an image (coefficient data). Forexample, the encoding information includes information regarding theblock size (TbW, TbH) of the block to be processed, informationregarding the size of the valid transform coefficient region in theblock to be processed (also referred to as valid transform coefficientregion information), and the like. The scan control unit 171 extractsthe information regarding the block size of the block to be processedand the valid transform coefficient region information from the acquiredencoding information.

The scan control unit 171 specifies the range of the valid transformcoefficient region (the position of the valid transform coefficientregion) in the block to be processed, based on the information. The scancontrol unit 171 controls the order of scan of the coefficient dataperformed by the encoding unit 172, based on the information. That is,the scan control unit 171 controls the order of scan so that, forexample, the encoding unit 172 skips the encoding of the coefficientdata of the sub-blocks in the invalid transform coefficient region andencodes the coefficient data of the sub-blocks in the valid transformcoefficient region as described above in <4-1: Scan Example of Method1>. The scan control unit 171 supplies a control command or the like tothe encoding unit 172 to control the order of scan of the encoding unit172.

The encoding unit 172 executes a process related to encoding. Forexample, the encoding unit 172 acquires encoding information and animage (coefficient data) input to the encoding apparatus 170. Inaddition, the encoding unit 172 acquires a control command or the likerelated to the scan supplied from the scan control unit 171. Theencoding unit 172 encodes the coefficient data in the order of scancontrolled by the scan control unit 171 and generates encoded data ofthe coefficient data. For example, the encoding unit 172 skips theencoding of the coefficient data related to the image in the sub-blocksof the invalid transform coefficient region and encodes the coefficientdata in the sub-blocks of the valid transform coefficient region asdescribed above in <4-1: Scan Example of Method 1>.

That is, the encoding unit 172 may skip the encoding of the coefficientdata in the invalid transform coefficient region and encode thecoefficient data in the valid transform coefficient region based on theinformation indicating the block size of the block to be processed andthe information indicating the range of the valid transformationcoefficient region.

In addition, the content of the coefficient data is optional. Forexample, the coefficient data may include valid transform coefficientflag information (codec_sub_block_flag) indicating the presence/absenceof the valid transform coefficient for each sub-block. Further, theencoding unit 172 may skip the encoding of the valid transformcoefficient flag information (codec_sub_block_flag) of each sub-block inthe invalid transform coefficient region and encode the valid transformcoefficient flag information (codec_sub_block_flag) of each sub-block inthe valid transform coefficient region.

In addition, for example, the coefficient data may further includevalidity flag information (sig_coeff_flag) indicating, for eachsub-block, the presence/absence of the valid transform coefficient witha value that is not 0 and may include a sign (coeff_sign_flag) and anabsolute value (abs) of each valid transform coefficient. Further, theencoding unit 172 may encode the validity flag information(sig_coeff_flag) of the sub-block in which the valid transformcoefficient flag information (codec_sub_block_flag) is encoded and mayencode the sign (coeff_sign_flag) and the absolute value (abs) of eachvalid transform coefficient of the sub-block in which the validity flaginformation (sig_coeff_flag) is true.

In addition, the encoding unit 172 also encodes the encoding informationand generates encoded data of the encoding information. The encodingunit 172 outputs the generated encoded data as a bitstream to theoutside of the encoding apparatus 170.

In such a way, the encoding apparatus 170 can reduce the unnecessaryinformation included in the bitstream (the encoded data of thecoefficient data of the sub-blocks in the invalid transform coefficientregion), and the reduction in the encoding efficiency can be suppressed.In addition, the encoding apparatus 170 can skip the encoding of theunnecessary information, and therefore, the encoding apparatus 170 cansuppress the increase in the load of the encoding process. The encodingapparatus 170 can suppress the increase in the cost, the circuit scale,the processing time, and the like (typically, the device can be moreinexpensively developed and manufactured, the device can be more easilydownsized, and the encoding can be performed faster).

<Flow of Encoding Process>

An example of a flow of the encoding process executed by the encodingapparatus 170 will be described with reference to a flow chart of FIG.18. Here, processing of one block will be described.

Once the encoding process is started, the scan control unit 171 of theencoding apparatus 170 acquires the encoding information and extractsthe transform block size (TbW, TbH) and the valid transform coefficientregion information (ZoW, ZoH) from the encoding information in stepS101.

In step S102, the scan control unit 171 controls the scan by theencoding unit 172 based on the information extracted in step S101(transform block size (TbW, TbH) and valid transform coefficient regioninformation (ZoW, ZoH)) to skip the scan of the sub-blocks in theinvalid transform coefficient region and scan the sub-blocks in thevalid transform count region as described above in, for example, <4-1:Scan Example of Method 1>.

In step S103, the encoding unit 172 scans each sub-block according tothe control and encodes the valid transform coefficient flag information(coded_sub_block_flag) of each sub-block. That is, the encoding unit 172skips the scan of the sub-blocks in the invalid transform coefficientregion and scans only the sub-blocks in the valid transform coefficientregion as described above in, for example, <4-1: Scan Example of Method1>. Therefore, only the valid transform coefficient flag information(coded_sub_block_flag) of each sub-block in the valid transformcoefficient region is encoded.

In step S104, the encoding unit 172 encodes the validity flaginformation (sig_coeff_flag) of each sub-block based on the value of thevalid transform coefficient flag information (coded_sub_block_flag)encoded in step S103. That is, the encoding unit 172 encodes thevalidity flag information (sig_coeff_flag) of each sub-block in whichthe valid transform coefficient flag information (coded_sub_block_flag)is true (for example, “1”).

In addition, the encoding unit 172 encodes the sign (coeff_sign_flag)and the absolute value (abs) of the transform coefficient according tothe value of the encoded validity flag information (sig_coeff_flag).That is, the encoding unit 172 encodes the sign (coeff_sign_flag) andthe absolute value (abs) of each transform coefficient of each sub-blockin which the validity flag information (sig_coeff_flag) is true (forexample, “1”).

That is, in the processes of step S103 and step S104, the encoding unit172 skips the encoding of the coefficient data related to the image ineach sub-block of the invalid transform coefficient region and encodesthe coefficient data in each sub-block of the valid transformcoefficient region as described above in, for example, <4-1: ScanExample of Method 1>. Further, the encoding unit 172 also encodes theencoding information. The encoding unit 172 outputs the encoded data asa bitstream. Once the process of step S104 is completed, the encodingprocess ends.

In such a way, the encoding apparatus 170 can reduce the unnecessaryinformation (the encoded data of the coefficient data of the sub-blocksin the invalid transform coefficient region) included in the bitstream,and the reduction in the encoding efficiency can be suppressed. Inaddition, the encoding apparatus 170 can skip the encoding of theunnecessary information, and therefore, the encoding apparatus 170 cansuppress the increase in the load of the encoding process. The encodingapparatus 170 can suppress the increase in the cost, the circuit scale,the processing time, and the like (typically, the device can be moreinexpensively developed and manufactured, the device can be more easilydownsized, and the encoding can be performed faster).

<4-3: Decoding> <Decoding Apparatus>

FIG. 19 is a block diagram illustrating an example of a mainconfiguration of a decoding apparatus as a mode of the image processingapparatus according to the present technique. A decoding apparatus 180illustrated in FIG. 19 is an apparatus that decodes encoded data of aninput image and that outputs coefficient data of the image. Asillustrated in FIG. 19, the decoding apparatus 180 includes a scancontrol unit 181 and a decoding unit 182.

The scan control unit 181 executes a process related to the control ofscan. For example, the scan control unit 181 acquires a bitstream(encoded data) input from the outside of the decoding apparatus 180. Thebitstream is, for example, generated by the encoding apparatus 170, andthe bitstream includes encoded data of coefficient data encoded in theorder of scan for skipping the encoding of the coefficient data of thesub-blocks in the invalid transform coefficient region that isunnecessary information and includes encoded data of encodinginformation. The scan control unit 181 extracts the block size (TbW,TbH) of the block to be processed and the valid transform coefficientregion information from the acquired bitstream.

The scan control unit 181 specifies the range of the valid transformcoefficient region (the position of the valid transform coefficientregion) in the block to be processed, based on the information. The scancontrol unit 181 reproduces the scan order in encoding (that is, scancontrol by the scan control unit 171), based on the information. Thescan control unit 181 supplies a control command or the like to thedecoding unit 182 and thereby controls the decoding unit 182 toreproduce the order of scan (that is, the order of scan for skipping theencoding of the coefficient data of the sub-blocks in the invalidtransform coefficient region and encoding the coefficient data of thesub-blocks in the valid transform coefficient region as described abovein, for example, <4-1: Scan Example of Method 1>).

The decoding unit 182 executes a process related to decoding. Forexample, the decoding unit 182 acquires a bitstream input to thedecoding apparatus 180. The bitstream includes the encoded data of thecoefficient data and the encoding information. In addition, the decodingunit 182 acquires a control command or the like related to scan suppliedfrom the scan control unit 181. The decoding unit 182 decodes thebitstream to obtain the coefficient data, based on the order of scan inencoding reproduced according to the control of the scan control unit181. For example, the decoding unit 182 skips decoding of the encodeddata including the encoded coefficient data related to the image in thesub-blocks of the invalid transform coefficient region and decodes theencoded data in the sub-blocks of the valid transform coefficient regionas described above in, for example, <4-1: Scan Example of Method 1>.

That is, the decoding unit 182 may skip the decoding of the encoded dataof the coefficient data in the invalid transform coefficient region anddecode the encoded data of the coefficient data in the valid transformcoefficient region, based on the information indicating the block sizeof the block to be processed and the information indicating the range ofthe valid transform coefficient region.

In addition, the content of the coefficient data is optional. Forexample, the coefficient data may include the valid transformcoefficient flag information (codec_sub_block_flag) indicating thepresence/absence of the valid transform coefficient for each sub-block.Further, the decoding unit 182 may decode the encoded data of the validtransform coefficient flag information (codec_sub_block_flag) of eachsub-block in the valid transform coefficient region.

In addition, for example, the coefficient data may further include thevalidity flag information (sig_coeff_flag) indicating, for eachsub-block, the presence/absence of the valid transform coefficient witha value that is not 0 and may include the sign (coeff_sign_flag) and theabsolute value (abs) of each valid transform coefficient. Further, thedecoding unit 182 may decode the encoded data of the validity flaginformation (sig_coeff_flag) of the sub-block for which the encoded dataof the valid transform coefficient flag information(codec_sub_block_flag) is decoded and may decode the encoded data of thesign (coeff_sign_flag) and the absolute value (abs) of each validtransform coefficient in the sub-block in which the validity flaginformation (sig_coeff_flag) is true.

For example, the decoding unit 182 decodes the encoded data of thecoefficient data included in the bitstream, to obtain the transmittedcoefficient data. In addition, the decoding unit 182 specifies thesub-blocks corresponding to the transmitted coefficient data based onthe order of scan controlled by the scan control unit 181 (that is, theorder of scan for skipping the encoding of the coefficient data of thesub-blocks in the invalid transform coefficient region and encoding thecoefficient data of the sub-blocks in the valid transform coefficientregion as described above in, for example, <4-1: Scan Example of Method1>). That is, the transmitted coefficient data is associated with thesub-blocks in the valid transform coefficient region. Further, thedecoding unit 182 restores the coefficient data of other sub-blocks,that is, the coefficient data not transmitted (for example, invalidtransform coefficients and the like). The decoding unit 182 generatesthe coefficient data of each block in such a way and outputs thecoefficient data to the outside of the decoding apparatus 180.

In such a way, the decoding apparatus 180 can correctly decode theencoded data generated in the order of scan for reducing the unnecessaryinformation included in the bitstream (the encoded data of thecoefficient data of the sub-blocks in the invalid transform coefficientregion). That is, the decoding apparatus 180 can suppress the reductionin the encoding efficiency. In addition, the decoding apparatus 180 canskip the decoding of the unnecessary information, and therefore, thedecoding apparatus 180 can suppress the increase in the load of thedecoding process. The decoding apparatus 180 can suppress the increasein the cost, the circuit scale, the processing time, and the like(typically, the device can be more inexpensively developed andmanufactured, the device can be more easily downsized, and the decodingcan be performed faster).

<Flow of Decoding Process>

An example of a flow of the encoding process executed by the decodingapparatus 180 will be described with reference to a flow chart of FIG.20. Here, processing of one block will be described.

Once the decoding process is started, in step S121, the scan controlunit 181 of the decoding apparatus 180 acquires the bitstream andextracts the transform block size (TbW, TbH) and the valid transformcoefficient region information (ZoW, ZoH) from the bitstream.

In step S122, the scan control unit 181 controls the decoding unit 182based on the information (transform block size (TbW, TbH) and validtransform coefficient region information (ZoW, ZoH)) extracted in stepS121 to reproduce the order of scan in encoding (the order of scan forskipping the scan of the sub-blocks in the invalid transform coefficientregion and scanning the sub-blocks in the valid transform count regionas described above in, for example, <4-1: Scan Example of Method 1>).

In step S123, the decoding unit 182 decodes the encoded data of thevalid transform coefficient flag information (coded_sub_block_flag) ofeach sub-block according to the order of scan in encoding, based on thecontrol.

For example, the decoding unit 182 decodes the encoded data of the validtransform coefficient flag information (coded_sub_block_flag) includedin the bitstream and obtains the transmitted valid transform coefficientflag information (coded_sub_block_flag). In addition, the decoding unit182 specifies the sub-blocks corresponding to the transmitted validtransform coefficient flag information (coded_sub_block_flag) based onthe order of scan controlled by the scan control unit 181 (that is, theorder of scan in encoding). Further, the decoding unit 182 restores thevalid transform coefficient flag information (coded_sub_block_flag) ofother sub-blocks, that is, the valid transform coefficient flaginformation (coded_sub_block_flag) not transmitted.

For example, in the case where the valid transform coefficient flaginformation (coded_sub_block_flag) of the sub-blocks in the validtransform coefficient region is obtained by decoding the bitstream, thedecoding unit 182 restores the valid transform coefficient flaginformation (coded_sub_block_flag) of the sub-blocks in the invalidtransform coefficient region that is another region. In such a way, thedecoding unit 182 restores the valid transform coefficient flaginformation (coded_sub_block_flag) of all of the sub-blocks of the blockto be processed.

In step S124, the decoding unit 182 decodes the validity flaginformation (sig_coeff_flag) of each sub-block, based on the value ofthe valid transform coefficient flag information (coded_sub_block_flag)decoded in step S123.

For example, the decoding unit 182 decodes the encoded data of thevalidity flag information (sig_coeff_flag) included in the bitstream andobtains the transmitted validity flag information (sig_coeff_flag). Inaddition, the decoding unit 182 specifies the sub-blocks correspondingto the transmitted validity flag information (sig_coeff_flag), based onthe decoding result of the valid transform coefficient flag information(coded_sub_block_flag). Further, the decoding unit 182 restores thevalidity flag information (sig_coeff_flag) of other sub-blocks, that is,the validity flag information (sig_coeff_flag) not transmitted.

Further, the decoding unit 182 decodes the sign (coeff_sign_flag) andthe absolute value (abs) of the transform coefficient included in eachsub-block, based on the value of the validity flag information(sig_coeff_flag) of each sub-block restored in such a way.

For example, the decoding unit 182 decodes the encoded data of the sign(coeff_sign_flag) and the absolute value (abs) of the transformcoefficient included in the bitstream and obtains the sign(coeff_sign_flag) and the absolute value (abs) of the transmittedtransform coefficient. In addition, the decoding unit 182 associates thesign (coeff_sign_flag) and the absolute value (abs) of the transmittedtransform coefficient with the sub-block in which the validity flaginformation (sig_coeff_flag) is true (for example, “1”). Further, thedecoding unit 182 restores the sign (coeff_sign_flag) and the absolutevalue (abs) of the transform coefficient included in another sub-block,that is, the sign (coeff_sign_flag) and the absolute value (abs) of thetransform coefficient not transmitted.

That is, in the processes of step S123 and step S124, the decoding unit182 skips the decoding of the encoded data including the encodedcoefficient data related to the image in each sub-block of the invalidtransform coefficient region and decodes the encoded data in eachsub-block of the valid transform coefficient region. The decoding unit182 outputs the coefficient data generated in such a way. Once theprocess of step S124 is completed, the decoding process ends.

In such a way, the decoding apparatus 180 can correctly decode theencoded data generated in the order of scan for reducing the unnecessaryinformation included in the bitstream (the encoded data of thecoefficient data of the sub-blocks in the invalid transform coefficientregion). That is, the decoding apparatus 180 can suppress the reductionin the encoding efficiency. In addition, the decoding apparatus 180 canskip the decoding of the unnecessary information, and therefore, thedecoding apparatus 180 can suppress the increase in the load of thedecoding process. The decoding apparatus 180 can suppress the increasein the cost, the circuit scale, the processing time, and the like(typically, the device can be more inexpensively developed andmanufactured, the device can be more easily downsized, and the decodingcan be performed faster).

<4-4: Syntax>

Although the skip of the sub-blocks in the invalid transform coefficientregion as described above may be realized by changing the order of scanas described above, the skip may be realized by using syntax.

FIG. 21 illustrates an example of the syntax in this case. Although theexample of the syntax is basically similar to the example of FIG. 6, therange of the scan is limited to the valid transform coefficient region((xS<=32) && (yS<=32)) in the if statement of the sixth stage from thetop. In such a way, the abovementioned scan control can easily berealized.

5. Second Embodiment <5-1: Scan Example of Method 1-1>

In the present embodiment the “method 1-1” will be described. Asdescribed above, in addition to the method 1 described above, the orderof scan in the valid transform coefficient region of the block to beprocessed is controlled in the method 1-1. In such a way, the scandirection of the sub-block units can be any direction, and a scandirection other than the diagonal direction can be realized. As aresult, the encoding and the decoding can be performed in, for example,a more appropriate scan order for the tendency of the distribution ofthe values of the coefficient data in the block to be processed.Therefore, the reduction in the encoding efficiency can be suppressed.

For example, the scan control unit 171 in the encoding apparatus 170 maycontrol the order of scan in the valid transform coefficient region instep S102, and the encoding unit 172 may scan and encode the sub-blocksin the valid transform coefficient region according to the control instep S103 and step S104. In such a way, the encoding apparatus 170 canfurther suppress the reduction in the encoding efficiency.

In addition, for example, the scan control unit 181 in the decodingapparatus 180 may control the order of scan in the valid transformcoefficient region in step S122, and the decoding unit 182 may use thescan of the sub-blocks in the valid transform coefficient regionaccording to the control to decode the encoded data in step S123 andstep S124. In such a way, the decoding apparatus 180 can furthersuppress the reduction in the encoding efficiency.

Specific examples of the control of the order of scan will be describedbelow.

<5-1-1: Method 1-1-1>

For example, the scan direction of the sub-block units in the validtransform coefficient region in the encoding and the decoding may bevertical or horizontal (method 1-1-1).

For example, in a case of FIG. 22, the size of the block 20 is 64×32. Inaddition, 32×32 on the left is a valid transform coefficient region(white part in FIG. 22), and 32×32 on the right is an invalid transformcoefficient region (gray part in FIG. 22). The scan is started from thesub-block 41 at the lower right corner of the valid transformcoefficient region. Further, only the sub-blocks in the valid transformcoefficient region are scanned as in the case of the example in FIG. 15.

However, in the case of the example in FIG. 22, the sub-block units arescanned in the vertical direction of FIG. 22. That is, the scan isstarted from the sub-block 41, and the sub-blocks in the same column arescanned from the bottom to the top. The sub-blocks in each column arescanned in a similar order in the vertical direction. Further, thecolumns are sequentially scanned one by one from the right column. Thatis, the sub-blocks are scanned in order from an arrow 201 to an arrow208.

For example, in a case where the prediction direction of the intraprediction is the horizontal direction, the values of the coefficientdata tend to significantly change in the vertical direction as a resultof the orthogonal transform. That is, the distribution of the values ofthe coefficient data tends to significantly change more (that is, thedeviation of the values becomes larger) in the vertical direction thanin the horizontal direction in this case. Therefore, as in the exampleof FIG. 22, the scan direction of the sub-block units in the encodingand the decoding can be set to the vertical direction to further utilizethe size of the deviation of the values, and the encoding efficiency canbe improved.

In addition, for example, although the shape, the valid transformcoefficient region, and the like of the block 20 in a case of FIG. 23are similar to those in the case of the example in FIG. 22, thesub-block units are scanned in the horizontal direction in FIG. 23. Thatis, the scan is started from the sub-block 41, and the sub-blocks in thesame row are scanned from the right to the left. The sub-blocks in eachrow are scanned in a similar order in the horizontal direction. Further,the rows are sequentially scanned one by one from the bottom row. Thatis, the sub-blocks are scanned in order from an arrow 211 to an arrow218.

For example, in a case where the prediction direction of the intraprediction is the vertical direction, the values of the coefficient datatend to significantly change in the horizontal direction as a result ofthe orthogonal transform. That is, the distribution of the values of thecoefficient data tends to significantly change more (that is, thedeviation of the values becomes larger) in the horizontal direction thanin the vertical direction in this case. Therefore, as in the example ofFIG. 23, the scan direction of the sub-block units in the encoding andthe decoding can be set to the horizontal direction to further utilizethe size of the deviation of the values, and the encoding efficiency canbe improved.

That is, the scan direction of the sub-block units in the validtransform coefficient region (for example, whether to set the scandirection to the vertical direction, the horizontal direction, or thediagonal direction) may be selected according to the predictiondirection of the intra prediction. In such a way, the encodingefficiency can be improved regardless of the prediction direction of theintra prediction.

For example, the size of the block 20 is 32×64 in a case of FIG. 24. Inaddition, upper 32×32 is a valid transform coefficient region (whitepart in FIG. 24), and lower 32×32 is an invalid transform coefficientregion (gray part in FIG. 24). The scan is started from the sub-block 41at the lower right corner of the valid transform coefficient region.Further, as in the case of the example in FIG. 16, only the sub-blocksin the valid transform coefficient region are scanned.

However, in the case of the example in FIG. 24, the sub-block units arescanned in the horizontal direction of FIG. 24. That is, the scan isstarted from the sub-block 41, and the sub-blocks of the same row arescanned from the right to the left. The sub-blocks in each row arescanned in the horizontal direction in a similar order. Further, therows are scanned one by one from the bottom row. That is, the sub-blocksare scanned in order from an arrow 231 to an arrow 238.

In the case where the intra prediction of the block to be processed isin the vertical direction, the scan direction of the sub-block units inthe encoding and the decoding can be set to the horizontal direction insuch a way to improve the encoding efficiency as in the example of FIG.23.

In addition, for example, although the shape, the valid transformcoefficient region, and the like of the block 20 in a case of FIG. 25are similar to those in the case of the example in FIG. 24, thesub-block units are scanned in the vertical direction in FIG. 25. Thatis, the scan is started from the sub-block 41, and the sub-blocks in thesame column are scanned from the bottom to the top. The sub-blocks ineach column are scanned in the vertical direction in a similar order.Further, the columns are sequentially scanned one by one from the rightcolumn. That is, the sub-blocks are scanned in order from an arrow 241to an arrow 248.

In the case where the intra prediction of the block to be processed isin the horizontal direction, the scan direction of the sub-block unitsin the encoding and the decoding can be set to the vertical direction insuch a way to improve the encoding efficiency as in the example of FIG.22.

For example, the size of the block 20 is 64×16 in a case of FIG. 26. Inaddition, 32×16 on the left is a valid transform coefficient region(white part in FIG. 26), and 32×16 on the right is an invalid transformcoefficient region (gray part in FIG. 26). The scan is started from thesub-block 41 on the lower right corner of the valid transformcoefficient region. Further, only the sub-blocks in the valid transformcoefficient region are scanned as in the examples described above.

In the case of the example in FIG. 26, the sub-block units are scannedin the vertical direction of FIG. 26. That is, the scan is started fromthe sub-block 41, and the sub-blocks in the same column are scanned fromthe bottom to the top. The sub-blocks in each column are scanned in thevertical direction in a similar order. Further, the columns aresequentially scanned one by one from the right column. That is, thesub-blocks are scanned in order from an arrow 261 to an arrow 268.

In the case where the intra prediction of the block to be processed isin the horizontal direction, the scan direction of the sub-block unitsin the encoding and the decoding can be set to the vertical direction insuch a way to improve the encoding efficiency as in the example of FIG.22.

In addition, for example, although the shape, the valid transformcoefficient region, and the like of the block 20 in a case of FIG. 27are similar to those in the case of the example in FIG. 26, thesub-block units are scanned in the horizontal direction in FIG. 27. Thatis, the scan is started from the sub-block 41, and the sub-blocks in thesame row are scanned from the right to the left. The sub-blocks in eachrow are scanned in the horizontal direction in a similar order. Further,the rows are sequentially scanned one by one from the bottom row. Thatis, the sub-blocks are scanned in order from an arrow 271 to an arrow274.

In the case where the intra prediction of the block to be processed isthe vertical direction, the scan direction of the sub-block units in theencoding and the decoding can be set to the horizontal direction in sucha way to improve the encoding efficiency as in the example of FIG. 23.

For example, the size of the block 20 is 16×64 in a case of FIG. 28. Inaddition, upper 16×32 is a valid transform coefficient region (whitepart in FIG. 28), and lower 16×32 is an invalid transform coefficientregion (gray part in FIG. 28). The scan is started from the sub-block 41at the lower right corner of the valid transform coefficient region.Further, only the sub-blocks in the valid transform coefficient regionare scanned as in the examples described above.

However, in the case of the example in FIG. 28, the sub-block units arescanned in the horizontal direction of FIG. 28. That is, the scan isstarted from the sub-block 41, and the sub-blocks in the same row arescanned from the right to the left. The sub-blocks in each row arescanned in the horizontal direction in a similar order. Further, therows are sequentially scanned one by one from the bottom row. That is,the sub-blocks are scanned in order from an arrow 281 to an arrow 288.

In the case where the intra prediction of the block to be processed isthe vertical direction, the scan direction of the sub-block units in theencoding and the decoding can be set to the horizontal direction in sucha way to improve the encoding efficiency as in the example of FIG. 24.

In addition, for example, although the shape, the valid transformcoefficient region, and the like of the block 20 in a case of FIG. 29are similar to those in the case in the example of FIG. 28, thesub-block units are scanned in the vertical direction in FIG. 29. Thatis, the scan is started from the sub-block 41, and the sub-blocks in thesame column are scanned from the bottom to the top. The sub-blocks ineach column are scanned in the vertical direction in a similar order.Further, the columns are sequentially scanned one by one from the rightcolumn. That is, the sub-blocks are scanned in order from an arrow 291to an arrow 294.

In the case where the intra prediction of the block to be processed isthe horizontal direction, the scan direction of the sub-block units inthe encoding and the decoding can be set to the vertical direction insuch a way to improve the encoding efficiency as in the example of FIG.25.

As described above, the shape and the size of the block and the validtransform coefficient region are optional. Obviously, the shape and thesize of the block and the valid transform coefficient region may beother than the shapes and the sizes in the examples described above.

<5-1-2: Method 1-1-2>

For example, in the encoding and the decoding, the scan direction of thesub-block units in the valid transform coefficient region may be adirection corresponding to the block shape (method 1-1-2).

The values of the coefficient data generally tend to gather in lowerfrequency components as a result of the orthogonal transform. That is,the value of the coefficient data increases toward the upper left cornerand decreases toward the lower right corner. Therefore, when the scan isstarted from high-frequency components and the high-frequency componentsare preferentially scanned as much as possible, the coefficients in thesame level are scanned in a growth ring shape.

However, the orthogonal transform is performed according to the shape ofthe transform block, and the low-frequency components of the transformcoefficients also tend to be distributed according to the shape of theblock. For example, in a case where the block shape is a square, thelow-frequency components in the same level are concentricallydistributed around the upper left. Further, in a case where the blockshape is a rectangle, the low-frequency components in the same level areelliptically distributed around the upper left.

Therefore, in a case where the scan direction of the sub-block units inthe encoding and the decoding is simply set to the diagonal directionregardless of the block shape, the scan direction matches the tendencyof the distribution of the values of the coefficient data in a squareblock. However, the scan direction does not match the tendency of thedistribution of the values of the coefficient data in a block withanother shape, such as, for example, a rectangle, and the encodingefficiency may be reduced.

Therefore, the scan direction of the sub-block units in the validtransform coefficient region can be set to a direction corresponding tothe block shape as described above. As a result, the scan can beperformed in a direction more suitable for the tendency of thedistribution of the values of the coefficient data, and the reduction inthe encoding efficiency can be suppressed.

Scan Example 1

For example, in a case where the shape of the valid transformcoefficient region is a vertically long rectangle, the high-frequencyregion is scanned with a slope of “−1,” and in a case where the scanreaches the left edge or the upper edge of the valid transformcoefficient region, the scan is performed with a slope at a ratio of“−height/width (−height of valid transform coefficient region/width ofvalid transform coefficient region).” In a case where it is difficult inpractice to accurately perform the scan with a slope of “height/width,”the scan in the vertical direction and the scan with a slope of −1 maybe combined to realize the scan.

For example, in a case of FIG. 30, the size of the block 20 is 16×64. Inaddition, upper 16×32 is a valid transform coefficient region (whitepart in FIG. 30), and lower 16×32 is an invalid transform coefficientregion (gray part in FIG. 30). The scan is started from the sub-block 41at the lower right corner of the valid transform coefficient region.Further, only the sub-blocks in the valid transform coefficient regionare scanned.

In the case of the example in FIG. 30, the scan is started from thesub-block 41, and the sub-blocks are scanned in order from an arrow 301to an arrow 313. That is, plural scan directions (for example, three ormore directions) are combined to perform the scan in this case (the scandirection is curved).

More specifically, the ratio of the width (blkWidth) and the height(blkHeight) of the valid transform coefficient region is 1 to 2 in thecase of FIG. 30. The horizontal direction of the valid transformcoefficient region in FIG. 30 will be defined as an x direction, thevertical direction in FIG. 30 will be defined as a y direction, and theupper left edge of the valid transform coefficient region will bedefined as an origin (x=0, y=0). In this case, the scan is firstperformed with a slope of y=−x (arrow 301 to arrow 304). Once the scanreaches x=0, the scan is performed in the y direction (arrow 305) aftermoving to the position of (height/2, 0). Further, once the scan reachesthe position of (height/2, height−width), the scan is performed with aslope of “−1.” Note that, in a case where y is not a multiple of 4 whenx=0, the scan moves to the unscanned maximum x with a slope of y=−x.

In such a way, the scan can be performed in a direction more suitablefor the tendency of the distribution of the values of the coefficientdata in the valid transform coefficient region in which the ratio of thewidth (blkWidth) and the height (blkHeight) of the valid transformcoefficient region is 1 to 2, and the reduction in the encodingefficiency can be suppressed.

In addition, for example, the size of the block 20 is 8×64 in a case ofFIG. 31. In addition, upper 8×32 is a valid transform coefficient region(white part in FIG. 31), and lower 8×32 is an invalid transformcoefficient region (gray part in FIG. 31). The scan is started from thesub-block 41 at the lower right corner of the valid transformcoefficient region. Further, only the sub-blocks in the valid transformcoefficient region are scanned.

In the case of the example in FIG. 31, the scan is started from thesub-block 41, and the sub-blocks are scanned in order from an arrow 321to an arrow 325. That is, plural scan directions (for example, three ormore directions) are combined to perform the scan in this case (scandirection is curved).

More specifically, the ratio of the width (blkWidth) and the height(blkHeight) of the valid transform coefficient region is 1 to 4 in thecase of FIG. 31. The horizontal direction of the valid transformcoefficient region in FIG. 31 will be defined as an x direction, thevertical direction in FIG. 31 will be defined as a y direction, and theupper left edge of the valid transform coefficient region will bedefined as an origin (x=0, y=0). In this case, the scan is firstperformed with a slope of y=−x (arrow 321 and arrow 322). Once the scanreaches x=0, the scan is performed in the y direction (arrow 323) aftermoving to the position of (height/4, 0). Further, once the scan reachesthe position of (height/4, height−width), the scan is performed with aslope of “−1.”

In such a way, the scan can be performed in a direction more suitablefor the tendency of the distribution of the values of the coefficientdata in the valid transform coefficient region in which the ratio of thewidth (blkWidth) and the height (blkHeight) of the valid transformcoefficient region is 1 to 4, and the reduction in the encodingefficiency can be suppressed.

In addition, for example, the size of the block 20 is 4×64 in a case ofFIG. 32. In addition, upper 4×32 is a valid transform coefficient region(white part in FIG. 32), and lower 4×32 is an invalid transformcoefficient region (gray part in FIG. 32). The scan is started from thesub-block 41 at the lower (lower right) corner of the valid transformcoefficient region. Further, only the sub-blocks in the valid transformcoefficient region are scanned.

In the case of the example in FIG. 32, there is one column of sub-blocksin the valid transform coefficient region. Therefore, the scan isstarted from the sub-block 41, and the sub-blocks are scanned from thebottom to the top as indicated by an arrow 331.

In such a way, the scan can be performed in a direction more suitablefor the tendency of the distribution of the values of the coefficientdata in the valid transform coefficient region including one column ofsub-blocks, and the reduction in the encoding efficiency can besuppressed.

Although the shape of the valid transform coefficient region is arectangle with the longitudinal direction in the vertical direction(that is, vertically long rectangle) in the case described above, thedescription similarly applies to a case where the shape of the validtransform coefficient region is a rectangle with the longitudinaldirection in the horizontal direction (that is, horizontally longrectangle).

That is, in the case where, for example, the shape of the validtransform coefficient region is a horizontally long rectangle, thehigh-frequency region is scanned with a slope of “−1,” and in the casewhere the scan reaches the left edge or the upper edge of the validtransform coefficient region, the scan is performed with a slope at aratio of “−height/width (−height of valid transform coefficientregion/width of valid transform coefficient region).” In a case where itis difficult in practice to accurately perform the scan with a slope of“height/width,” the scan in the vertical direction and the scan with aslope of −1 may be combined to realize the scan.

For example, in a case where the ratio of the width (blkWidth) and theheight (blkHeight) of the valid transform coefficient region is 2 to 1,the scan is first performed with a slope of y=−x. Once the scan reachesy=0, the scan is performed in the x direction after moving to theposition of (0, width/2). Further, once the scan reaches the position of(width−height, width/2), the scan is performed with a slope of “−1.”Note that, in a case where x is not a multiple of 4 when y=0, the scanmoves to the unscanned maximum y with a slope of y=−x.

In such a way, the scan can be performed in a direction more suitablefor the tendency of the distribution of the values of the coefficientdata in the valid transform coefficient region in which the ratio of thewidth (blkWidth) and the height (blkHeight) of the valid transformcoefficient region is 2 to 1, and the reduction in the encodingefficiency can be suppressed.

Further, in a case where, for example, the ratio of the width (blkWidth)and the height (blkHeight) of the valid transform coefficient region is4 to 1, the scan is first performed with a slope of y=−x. Once the scanreaches y=0, the scan is performed in the x direction after moving tothe position of (0, width/4). Further, once the scan reaches theposition of (width−height, width/4), the scan is performed with a slopeof “−1.”

In such a way, the scan can be performed in a direction more suitablefor the tendency of the distribution of the values of the coefficientdata in the valid transform coefficient region in which the ratio of thewidth (blkWidth) and the height (blkHeight) of the valid transformcoefficient region is 4 to 1, and the reduction in the encodingefficiency can be suppressed.

Further, in a case where, for example, there is one row of sub-blocks inthe valid transform coefficient region, the scan is started from thesub-block 41, and the sub-blocks are scanned from the right to the left.

In such a way, the scan can be performed in a direction more suitablefor the tendency of the distribution of the values of the coefficientdata in the valid transform coefficient region including one row ofsub-blocks, and the reduction in the encoding efficiency can besuppressed.

Scan Example 2

Obviously, the setting method of the scan direction corresponding to theblock shape is optional, and the setting method is not limited to theexample described above. For example, in a case of a valid transformcoefficient region of width 16×height 32, a method as illustrated in Aof FIG. 33 may be used to derive the order of scan.

That is, the height (blkheight) of the valid transform coefficientregion is represented by 2n where n is the width (blkwidth) of the validtransform coefficient region. In this case, the ellipse can be expressedas in the following Equation (1).

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 1} \right\rbrack & \; \\{{\frac{x^{2}}{n^{2}} + \frac{y^{2}}{4n^{2}}} = 1} & (1)\end{matrix}$

Here, n when the ellipse passes through the vertex on the upper left isobtained for each sub-block. Here, n is derived as in the followingEquation (2).

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 2} \right\rbrack & \; \\{n = \sqrt{x^{2} + \frac{y^{2}}{4}}} & (2)\end{matrix}$

In the block 20 as illustrated in FIG. 34, n of each sub-block is as inB of FIG. 33. For example, n=12 is elliptic as depicted by a solid curvein FIG. 34, and n=18.4 is elliptic as depicted by a dotted curve in FIG.34. Therefore, the sub-blocks are scanned in descending order of n.Here, in a case where the values of n are the same, either one can beselected.

In this case, when, for example, the ellipse passing through theposition (16, 0) and the position (0, 32) is approximated as a straightline, y=−2x+32 is obtained. Therefore, the scan is basically carried outwith a slope of −2. For example, the scan may be carried out asindicated by arrows illustrated in FIG. 35. In such a way, the scan canbe performed in a direction more suitable for the tendency of thedistribution of the values of the coefficient data in the block in whichthe ratio of the width and the height of the valid transform coefficientregion is 1 to 2, and the reduction in the encoding efficiency can besuppressed.

Further, in a case of, for example, a valid transform coefficient regionof width 8×height 32, a method as illustrated in A of FIG. 36 may beused to derive the order of scan. In this case, the height (blkheight)of the valid transform coefficient region is represented by 4 n where nis the width (blkwidth) of the valid transform coefficient region. Inthis case, the ellipse can be expressed as in the following Equation(3).

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 3} \right\rbrack & \; \\{{\frac{x^{2}}{n^{2}} + \frac{y^{2}}{16n^{2}}} = 1} & (3)\end{matrix}$

Here, n when the ellipse passes through the vertex on the upper left isobtained for each sub-block. Here, n is derived as in the followingEquation (4).

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 1} \right\rbrack & \; \\{n = \sqrt{x^{2} + \frac{y^{2}}{16}}} & (1)\end{matrix}$

In the block 20 as illustrated in FIG. 37, n of each sub-block is as inB of FIG. 36. For example, n=4 is elliptic as depicted by a solid curvein FIG. 37, and n=8.1 is elliptic as depicted by a dotted curve in FIG.37. Therefore, the sub-blocks are scanned in descending order of n.Here, in a case where the values of n are the same, either one can beselected.

In this case, when, for example, the ellipse passing through theposition (8, 0) and the position (0, 32) is approximated as a straightline, y=−4x+32 is obtained. Therefore, the scan is basically carried outwith a slope of −4. For example, the scan may be carried out asindicated by arrows illustrated in FIG. 38. In such a way, the scan canbe performed in a direction more suitable for the tendency of thedistribution of the values of the coefficient data in the block in whichthe ratio of the width and the height of the valid transform coefficientregion is 1 to 4, and the reduction in the encoding efficiency can besuppressed.

Note that it is sufficient if, in a case where the width of the validtransform coefficient region is equivalent to one sub-block, that is, ina case where the valid transform coefficient region includes one columnof sub-blocks, the scan is performed in an order similar to the examplein FIG. 32.

Further, it is sufficient if, in a case where the valid transformcoefficient region is a horizontally long rectangle, the scan isbasically similar to that in the example described above.

<5-1-3: Method 1-1-3>

For example, the sub-blocks in the valid transform coefficient regionmay be grouped to control the scan in the encoding and the decoding(method 1-1-3).

For example, as illustrated in FIG. 39, the sub-blocks in the validtransform region are put into 1×2 groups in the block 20 including avalid transform coefficient region of 16×32. A sub-block group 401indicated by a mesh pattern is a region with an aspect ratio of 1 to 2including one sub-block horizontally and two sub-blocks vertically.Although only the upper left edge is illustrated in FIG. 39, all of thesub-blocks in the valid transform coefficient region are actuallygrouped. That is, the sub-block groups are formed in the entire validtransform coefficient region. In each sub-block group, the sub-blockunit is scanned in the direction from the bottom to the top. Inaddition, the sub-block group units are scanned in the direction fromthe high-frequency component side to the low-frequency component side.That is, the sub-blocks in the valid transform coefficient region arescanned in order from an arrow 411 to an arrow 426.

In addition, for example, the sub-blocks in the valid transform regionare put into 1×4 groups in the block 20 including a valid transformcoefficient region of 8×32 as illustrated in FIG. 40. The sub-blockgroup 401 indicated by a mesh pattern is a region with an aspect ratioof 1 to 4 including one sub-block horizontally and four sub-blocksvertically. Although only the upper left edge is illustrated in FIG. 40,all of the sub-blocks in the valid transform coefficient region areactually grouped. That is, the sub-block groups are formed in the entirevalid transform coefficient region. In each sub-block group, thesub-block unit is scanned in the direction from the bottom to the top.In addition, the sub-block group units are scanned in the direction fromthe high-frequency component side to the low-frequency component side.That is, the sub-blocks in the valid transform coefficient region arescanned in order from an arrow 431 to an arrow 434.

As described above, the sub-blocks in the valid transform coefficientregion may be grouped to control the scan. Further, in that case, thegroups of the sub-blocks may be formed to provide the same aspect ratioas the aspect ratio of the valid transform coefficient region.

In such a way, the scan can be performed in a direction more suitablefor the tendency of the distribution of the values of the coefficientdata, and the reduction in the encoding efficiency can be suppressed.

Note that, although the valid transform coefficient regions in variousshapes are described in the first embodiment and the second embodiment,the shapes of the valid transform coefficient region are not limited tothe examples, and the shapes are optional. For example, the shape of thevalid transform coefficient region may be different from the blockshape. For example, as illustrated in FIG. 41, the valid transformcoefficient region may be set to a rectangle even though the block 20 isa square. In the case of the example in FIG. 41, a valid transformcoefficient region of 32×16 is set in the block 20 of 64×64. In thiscase, the scan of the sub-block units is started from the sub-block 41,and the scan is performed in order from an arrow 441 to an arrow 451.

6. Third Embodiment <Scan Example of Method 1-2>

In the present embodiment, the “method 1-2” will be described. Theprocess (Zero Out) of setting the high-frequency region coefficients ofthe block in a size larger than 32×32 to 0 described above may beapplied not only to the luminance components, but also to the chromacomponents. This can further suppress the reduction in the encodingefficiency.

Further, in that case, the present technique may also be applied to thechroma components to which the process is applied. For example, themethod 1 may be applied to the chroma components as in the case of theluminance components. In such a way, the reduction in the encodingefficiency can be suppressed. In addition, the increase in the load ofencoding and decoding can be suppressed, and the increase in the cost,the circuit scale, the processing time, and the like can be suppressed.In addition, for example, the method 1-1 may also be applied to thechroma components as in the case of the luminance components. Forexample, one of the methods 1-1-1 to 1-1-3 may be applied to the chromacomponents. In such a way, the reduction in the encoding efficiency canbe suppressed.

Note that, in that case, the block size of the chroma components may notbe the same as the block size of the luminance components. For example,while the block 20 of the luminance components is 64×64 as in theexample of FIG. 14, a block 460 of the chroma components may be 32×32 asillustrated in FIG. 42. Assuming that the scan is started from asub-block 461 at the lower right corner of the valid transformcoefficient region in the block 460 of the chroma components, thesub-block units are scanned in order from an arrow 471 to an arrow 477as illustrated in FIG. 42. That is, in this case as well, the scan ofthe chroma components is performed similarly to the scan of theluminance components. In such a way, the scan method can be uniform forthe luminance components and the chroma components, and the scan can bemore easily controlled.

For example, the scan control unit 171 in the encoding apparatus 170 mayalso control the order of scan for the chroma components in step S102 asin the case of the luminance components, and the encoding unit 172 mayscan and encode the sub-blocks in step S103 and step S104 according tothe control. In such a way, the encoding apparatus 170 can also suppressthe reduction in the encoding efficiency for the chroma components.

In addition, for example, the scan control unit 181 in the decodingapparatus 180 may also control the order of scan for the chromacomponents in step S122 as in the case of the luminance components, andthe decoding unit 182 may use the scan of each sub-block to decode theencoded data for the chroma components in step S123 and step S124according to the control as in the case of the luminance components. Insuch a way, the decoding apparatus 180 can also further suppress thereduction in the encoding efficiency for the chroma components.

7. Fourth Embodiment <7-1: Image Encoding Apparatus>

The present technique described above can be applied to any apparatuses,devices, systems, and the like. For example, the present techniquedescribed above can be applied to an image encoding apparatus thatencodes image data.

FIG. 43 is a block diagram illustrating an example of the configurationof the image encoding apparatus that is a mode of an image processingapparatus according to the present technique. An image encodingapparatus 500 illustrated in FIG. 43 is an apparatus that encodes imagedata of moving images. For example, the image encoding apparatus 500 isprovided with the techniques described in NPL 1 to NPL 4, and the imageencoding apparatus 500 uses a method compliant with the standarddescribed in any of the abovementioned pieces of literature to encodethe image data of the moving images.

Note that FIG. 43 illustrates main processing units, flows of data, andthe like, and FIG. 43 may not illustrate everything. That is, in theimage encoding apparatus 500, there may be processing units notillustrated as blocks in FIG. 43, and there may be flows of processesand data not indicated by arrows or the like in FIG. 43. This similarlyapplies to other drawings describing the processing units and the likein the image encoding apparatus 500.

As illustrated in FIG. 43, the image encoding apparatus 500 includes acontrol unit 501, a rearrangement buffer 511, a computation unit 512, anorthogonal transform unit 513, a quantization unit 514, an encoding unit515, an accumulation buffer 516, an inverse quantization unit 517, aninverse orthogonal transform unit 518, a computation unit 519, anin-loop filter unit 520, a frame memory 521, a prediction unit 522, anda rate control unit 523.

<Control Unit>

The control unit 501 divides moving image data held in the rearrangementbuffer 511 into blocks in a unit of processing (such as CU, PU, andtransform block), based on the block size of the unit of processingdesignated from the outside or designated in advance. In addition, thecontrol unit 501 determines encoding parameters (such as headerinformation Hinfo, prediction mode information Pinfo, transforminformation Tinfo, and filter information Finfo) to be supplied to eachblock, based on, for example, RDO (Rate-Distortion Optimization).

The details of the encoding parameters will be described later. Once thecontrol unit 501 determines the encoding parameters, the control unit501 supplies the encoding parameters to each block. Specifically, theencoding parameters are supplied as follows.

The header information Hinfo is supplied to each block. The predictionmode information Pinfo is supplied to the encoding unit 515 and theprediction unit 522. The transform information Tinfo is supplied to theencoding unit 515, the orthogonal transform unit 513, the quantizationunit 514, the inverse quantization unit 517, and the inverse orthogonaltransform unit 518. The filter information Finfo is supplied to thein-loop filter unit 520.

<Rearrangement Buffer>

Each field (input image) of the moving image data is input to the imageencoding apparatus 500 in the order of reproduction (order of display).The rearrangement buffer 511 acquires the input images in the order ofreproduction (order of display) and holds (stores) the input images. Therearrangement buffer 511 rearranges the input images in the order ofencoding (order of decoding) or divides the input images into blocks inthe unit of processing based on the control of the control unit 501. Therearrangement buffer 511 supplies the processed input images to thecomputation unit 512. In addition, the rearrangement buffer 511 alsosupplies the input images (original images) to the prediction unit 522and the in-loop filter unit 520.

<Computation Unit>

The computation unit 512 receives an image I corresponding to the blocksin the unit of processing and a predicted image P supplied from theprediction unit 522. The computation unit 512 subtracts the predictedimage P from the image I as indicated in the following Equation (5) toderive a predicted residual resi and supplies the predicted residualresi to the orthogonal transform unit 513.

[Math. 5]

resi=I−P  (5)

<Orthogonal Transform Unit>

The orthogonal transform unit 513 receives the predicted residual resisupplied from the computation unit 512 and the transform informationTinfo supplied from the control unit 501. The orthogonal transform unit513 applies an orthogonal transform to the predicted residual resi basedon the transform information Tinfo to derive a transform coefficientcoef. The orthogonal transform unit 513 supplies the obtained transformcoefficient coef to the quantization unit 514.

<Quantization Unit>

The quantization unit 514 receives the transform coefficient coefsupplied from the orthogonal transform unit 513 and the transforminformation Tinfo supplied from the control unit 501 and scales(quantizes) the transform coefficient coef based on the transforminformation Tinfo. Note that the rate of the quantization is controlledby the rate control unit 523. The quantization unit 514 supplies thequantized transform coefficient obtained by the quantization, that is, aquantization transform coefficient level qcoef to the encoding unit 515and the inverse quantization unit 517.

<Encoding Unit>

The encoding unit 515 receives the quantization transform coefficientlevel qcoef supplied from the quantization unit 514, the variousencoding parameters (such as header information Hinfo, prediction modeinformation Pinfo, transform information Tinfo, and filter informationFinfo) supplied from the control unit 501, information regarding afilter, such as a filter coefficient, supplied from the in-loop filterunit 520, and information regarding an optimal prediction mode suppliedfrom the prediction unit 522. The encoding unit 515 performsvariable-length encoding (for example, arithmetic coding) of thequantization transform coefficient level qcoeff to generate a bit string(encoded data).

In addition, the encoding unit 515 derives residual information Rinfofrom the quantization transform coefficient level qoeff and encodes theresidual information Rinfo to generate a bit string.

Further, the encoding unit 515 includes the information regarding thefilter supplied from the in-loop filter unit 520 in the filterinformation Finfo and includes the information regarding the optimalprediction mode supplied from the prediction unit 522 in the predictionmode information Pinfo. The encoding unit 515 then encodes the variousencoding parameters (such as header information Hinfo, prediction modeinformation Pinfo, transform information Tinfo, and filter informationFinfo) and generates bit strings.

In addition, the encoding unit 515 multiplexes the bit strings ofvarious types of information generated in such a way to generate encodeddata. The encoding unit 515 supplies the encoded data to theaccumulation buffer 516.

<Accumulation Buffer>

The accumulation buffer 516 temporarily holds the encoded data obtainedby the encoding unit 515. The accumulation buffer 516 outputs the heldencoded data as, for example, a bitstream or the like to the outside ofthe image encoding apparatus 500 at a predetermined timing. For example,the encoded data is transmitted to the decoding side through anyrecording medium, any transmission medium, any information processingapparatus, or the like. That is, the accumulation buffer 516 is also atransmission unit that transmits the encoded data (bitstream).

<Inverse Quantization Unit>

The inverse quantization unit 517 executes a process regarding inversequantization. For example, the inverse quantization unit 517 receivesthe quantization transform coefficient level qcoeff supplied from thequantization unit 514 and the transform information Tinfo supplied fromthe control unit 501 and scales (inverse quantization) the value of thequantization transform coefficient level based on the transforminformation Tinfo. Note that the inverse quantization is an oppositeprocess of the quantization executed by the quantization unit 514. Theinverse quantization unit 517 supplies a transform coefficient coefIobtained by the inverse quantization to the inverse orthogonal transformunit 518.

<Inverse Orthogonal Transform Unit>

The inverse orthogonal transform unit 518 executes a process regardingan inverse orthogonal transform. For example, the inverse orthogonaltransform unit 518 receives the transform coefficient coefI suppliedfrom the inverse quantization unit 517 and the transform informationTinfo supplied from the control unit 501 and applies an inverseorthogonal transform to the transform coefficient coefI based on thetransform information Tinfo to derive a predicted residual resiI. Notethat the inverse orthogonal transform is an opposite process of theorthogonal transform performed by the orthogonal transform unit 513. Theinverse orthogonal transform unit 518 supplies the predicted residualresiI obtained by the inverse orthogonal transform to the computationunit 519. Note that the inverse orthogonal transform unit 518 is similarto an inverse orthogonal transform unit (described later) on thedecoding side, and the description for the decoding side (describedlater) can be applied to the inverse orthogonal transform unit 518.

<Computation Unit>

The computation unit 519 receives the predicted residual resiI suppliedfrom the inverse orthogonal transform unit 518 and the predicted image Psupplied from the prediction unit 522. The computation unit 519 adds thepredicted residual resiI and the predicted image P corresponding to thepredicted residual resiI to derive a local decoded image Rlocal. Thecomputation unit 519 supplies the derived local decoded image Rlocal tothe in-loop filter unit 520 and the frame memory 521.

<In-Loop Filter Unit>

The in-loop filter unit 520 executes a process regarding an in-loopfilter process. For example, the in-loop filter unit 520 receives thelocal decoded image Rlocal supplied from the computation unit 519, thefilter information Finfo supplied from the control unit 501, and theinput image (original image) supplied from the rearrangement buffer 511.Note that the information to be input to the in-loop filter unit 520 isoptional, and information other than the information may be input. Forexample, information, such as a prediction mode, motion information, anencoding amount target value, a quantization parameter QP, a picturetype, and blocks (such as CU and CTU), may be input to the in-loopfilter unit 520 as necessary.

The in-loop filter unit 520 appropriately applies a filter process tothe local decoded image Rlocal based on the filter information Finfo.The in-loop filter unit 520 also uses the input image (original image)and other input information in the filter process as necessary.

For example, the in-loop filter unit 520 sequentially applies fourin-loop filters including a bilateral filter, a deblocking filter (DBF(DeBlocking Filter)), an adaptive offset filter (SAO (Sample AdaptiveOffset)), and an adaptive loop filter (ALF (Adaptive Loop Filter)), inthis order. Note that the filters to be applied and the order ofapplying the filters are optional and can appropriately be selected.

Obviously, the filter process executed by the in-loop filter unit 520 isoptional, and the filter process is not limited to the example. Forexample, the in-loop filter unit 520 may apply a Wiener filter or thelike.

The in-loop filter unit 520 supplies the filtered local decoded imageRlocal to the frame memory 521. Note that, in a case of transmitting,for example, information regarding the filter, such as a filtercoefficient, to the decoding side, the in-loop filter unit 520 suppliesthe information regarding the filter to the encoding unit 515.

<Frame Memory>

The frame memory 521 executes a process regarding storage of datarelated to an image. For example, the frame memory 521 receives thelocal decoded image Rlocal supplied from the computation unit 519 or thefiltered local decoded image Rlocal supplied from the in-loop filterunit 520 and holds (stores) the local decoded image Rlocal. In addition,the frame memory 521 uses the local decoded image Rlocal to reconstructthe decoded image R of each picture unit and holds the decoded image R(stores the decoded image R in a buffer of the frame memory 521). Theframe memory 521 supplies the decoded image R (or part of the decodedimage R) to the prediction unit 522 according to a request from theprediction unit 522.

<Prediction Unit>

The prediction unit 522 executes a process regarding generation of apredicted image. For example, the prediction unit 522 receives theprediction mode information Pinfo supplied from the control unit 501,the input image (original image) supplied from the rearrangement buffer511, and the decoded image R (or part of the decoded image R) read fromthe frame memory 521. The prediction unit 522 uses the prediction modeinformation Pinfo and the input image (original image) to execute aprediction process, such as inter prediction and intra prediction, andrefers to the decoded image R as a reference image to performprediction. The prediction unit 522 executes a motion compensationprocess based on the prediction result to generate a predicted image P.The prediction unit 522 supplies the generated predicted image P to thecomputation unit 512 and the computation unit 519. In addition, theprediction unit 522 supplies information regarding a prediction modeselected in the process, that is, an optimal prediction mode, to theencoding unit 515 as necessary.

<Rate Control Unit>

The rate control unit 523 executes a process regarding rate control. Forexample, the rate control unit 523 controls the rate of the quantizationoperation of the quantization unit 514 to prevent an overflow or anunderflow, based on the amount of encoding of the encoded dataaccumulated in the accumulation buffer 516.

<Application of Present Technique>

In the image encoding apparatus 500 configured as described above, theencoding apparatus 170 (FIG. 17) may be applied as the encoding unit515, for example. That is, the encoding unit 515 may include the scancontrol unit 171 and the encoding unit 172, and various types of scancontrol described in <3. Concept> to <6. Third Embodiment> may beapplied to the encoding performed by the encoding unit 172.

According to the configuration, the encoding unit 515 can reduce theunnecessary information included in the bitstream (the encoded data ofthe coefficient data of the sub-blocks in the invalid transformcoefficient region) and suppress the reduction in the encodingefficiency as in the case of the encoding apparatus 170. In addition,encoding of the unnecessary information can be skipped, and therefore,the increase in the load of the encoding process can be suppressed, andthe increase in the cost, the circuit scale, the processing time, andthe like can be suppressed.

Therefore, the image encoding apparatus 500 can suppress the reductionin the encoding efficiency. In addition, the image encoding apparatus500 can suppress the increase in the load of the encoding process andsuppress the increase in the cost, the circuit scale, the processingtime, and the like.

<Flow of Image Encoding Process>

Next, flows of processes executed by the image encoding apparatus 500will be described. First, an example of a flow of an image encodingprocess will be described with reference to a flow chart of FIG. 44.

Once the image encoding process is started, the rearrangement buffer 511rearranges the order of the frames of the moving image data controlledand input by the control unit 501 from the order of display to the orderof encoding in step S301.

In step S302, the control unit 501 sets the unit of processing for theinput image held by the rearrangement buffer 511 (divides the inputimage into blocks).

In step S303, the control unit 501 determines (sets) the encodingparameters of the input image held by the rearrangement buffer 511.

In step S304, the prediction unit 522 executes the prediction process togenerate the predicted image and the like of the optimal predictionmode. For example, in the prediction process, the prediction unit 522performs the intra prediction to generate the predicted image and thelike of the optimal intra prediction mode and performs the interprediction to generate the predicted image and the like of the optimalinter prediction mode. The prediction unit 522 selects the optimalprediction mode from them based on the cost function value and the like.

In step S305, the computation unit 512 computes the difference betweenthe input image and the predicted image of the optimal mode selected inthe prediction process of step S504. That is, the computation unit 512generates the predicted residual resi of the input image and thepredicted image. The amount of data of the predicted residual resiobtained in such a way is reduced compared to the original image data.Therefore, the amount of data can be compressed compared to the case inwhich the image is encoded as it is.

In step S306, the orthogonal transform unit 513 applies the orthogonaltransform process to the predicted residual resi generated in theprocess of step S305 to derive the transform coefficient coef.

In step S307, the quantization unit 514 quantizes the transformcoefficient coef obtained in the process of step S306 by, for example,using the quantization parameters calculated by the control unit 501 andderives the quantization transform coefficient level qcoef.

In step S308, the inverse quantization unit 517 uses the characteristicscorresponding to the characteristics of the quantization of step S307 toapply inverse quantization to the quantization transform coefficientlevel qcoef generated in the process of step S307 and derives thetransform coefficient coefI.

In step S309, the inverse orthogonal transform unit 518 uses the methodcorresponding to the orthogonal transform process of step S306 to applyan inverse orthogonal transform to the transform coefficient coefIobtained in the process of step S308 and derives the predicted residualresiI. Note that the inverse orthogonal transform process is similar toan inverse orthogonal transform process (described later) executed onthe decoding side, and the description for the decoding side (describedlater) can be applied to the inverse orthogonal transform process ofstep S309.

In step S310, the computation unit 519 adds the predicted image obtainedin the prediction process of step S304 to the predicted residual resiIderived in the process of step S309, to generate the decoded image thatis locally decoded.

In step S311, the in-loop filter unit 520 applies an in-loop filterprocess to the decoded image that is locally decoded and derived in theprocess of step S310.

In step S312, the frame memory 521 stores the decoded image that islocally decoded and derived in the process of step S310 and the decodedimage that is locally decoded filtered in step S312.

In step S313, the encoding unit 515 encodes the quantization transformcoefficient level qcoef obtained in the process of step S307. Forexample, the encoding unit 515 uses the arithmetic coding or the like toencode the quantization transform coefficient level qcoef that isinformation regarding the image and generates the encoded data. Inaddition, the encoding unit 515 encodes various encoding parameters(header information Hinfo, prediction mode information Pinfo, transforminformation Tinfo) at this point. Further, the encoding unit 515 derivesthe residual information RInfo from the quantization transformcoefficient level and encodes the residual information RInfo.

In step S314, the accumulation buffer 516 accumulates the encoded dataobtained in such a way and outputs the encoded data as, for example, abitstream to the outside of the image encoding apparatus 500. Thebitstream is transmitted to the decoding side through, for example, thetransmission path or the recording medium. In addition, the rate controlunit 523 performs the rate control as necessary.

Once the process of S314 is completed, the image encoding process ends.

In the encoding process executed in step S313 of the image encodingprocess, the order of scan of the valid transform coefficient region maybe controlled as described above. That is, the encoding process asdescribed with reference to the flow chart of FIG. 18 may be executed inthis encoding process. In such a way, the image encoding apparatus 500can further suppress the reduction in the encoding efficiency.

<7-2: Image Decoding Apparatus>

In addition, the present technique described above can be applied to,for example, an image decoding apparatus that decodes encoded dataincluding encoded image data.

FIG. 45 is a block diagram illustrating an example of a configuration ofthe image decoding apparatus that is a mode of the image processingapparatus according to the present technique. An image decodingapparatus 600 illustrated in FIG. 45 is an apparatus that decodesencoded data in which a predicted residual of an image and a predictedimage of the image is encoded as in AVC or HEVC. For example, the imagedecoding apparatus 600 is provided with the techniques described in NPL1 to NPL 4, and the image decoding apparatus 600 uses a method compliantwith a standard described in any of the abovementioned pieces ofliterature to decode the encoded data in which the image data of movingimages is encoded. For example, the image decoding apparatus 600 decodesthe encoded data (bitstream) generated by the image encoding apparatus500.

Note that FIG. 45 illustrates main processing units, flows of data, andthe like, and FIG. 45 may not illustrate everything. That is, in theimage decoding apparatus 600, there may be processing units notillustrated as blocks in FIG. 45, and there may be flows of processes ordata not indicated by arrows or the like in FIG. 45. This similarlyapplies to other drawings describing the processing units and the likein the image decoding apparatus 600.

In FIG. 45, the image decoding apparatus 600 includes an accumulationbuffer 611, a decoding unit 612, an inverse quantization unit 613, aninverse orthogonal transform unit 614, a computation unit 615, anin-loop filter unit 616, a rearrangement buffer 617, a frame memory 618,and a prediction unit 619. Note that the prediction unit 619 includes anintra prediction unit and an inter prediction unit not illustrated. Theimage decoding apparatus 600 is an apparatus that decodes the encodeddata (bitstream) to generate moving image data.

<Accumulation Buffer>

The accumulation buffer 611 acquires the bitstream input to the imagedecoding apparatus 600 and holds (stores) the bitstream. Theaccumulation buffer 611 supplies the accumulated bitstream to thedecoding unit 612 at a predetermined timing or in a case where, forexample, a predetermined condition is met.

<Decoding Unit>

The decoding unit 612 executes a process regarding decoding of an image.For example, the decoding unit 612 receives the bitstream supplied fromthe accumulation buffer 611. The decoding unit 612 appliesvariable-length decoding to syntax values of syntax elements from thebit string according to the definition of a syntax table and derivesparameters.

The syntax elements and the parameters derived from the syntax values ofthe syntax elements include information, such as, for example, headerinformation Hinfo, prediction mode information Pinfo, transforminformation Tinfo, residual information Rinfo, and filter informationFinfo. That is, the decoding unit 612 parses (analyzes and acquires) theinformation from the bitstream. The information will be described below.

<Header Information Hinfo>

The header information Hinfo includes, for example, header information,such as VPS (Video Parameter Set)/SPS (Sequence ParameterSet)/PPS(Picture Parameter Set)/SH (slice header). The header information Hinfoincludes, for example, information defining image size (width PicWidth,height PicHeight), bit depth (luminance bitDepthY, chroma bitDepthC),chroma array type ChromaArrayType, maximum value MaxCUSize/minimum valueMinCUSize of CU size, maximum depth MaxQTDepth/minimum depth MinQTDepthof quad-tree partition (also referred to as Quad-tree partition),maximum depth MaxBTDepth/minimum depth MinBTDepth of binary-treepartition (Binary-tree partition), maximum value MaxTSSize of transformskip block (also referred to as a maximum transform skip block size),on/off flag of each encoding tool (also referred to as enabled flag),and the like.

An example of the on/off flag of the encoding tool included in theheader information Hinfo includes the following on/off flag related tothe transform and the quantization process. Note that the on/off flag ofthe encoding tool can also be interpreted as a flag indicating whetheror not the encoded data includes the syntax related to the encodingtool. Further, in a case where the value of the on/off flag is 1 (true),the encoding tool can be used. In a case where the value of the on/offflag is 0 (false), the encoding tool cannot be used. Note that theinterpretation of the flag values may be opposite.

Cross-component prediction enabled flag (ccp_enabled_flag): flaginformation indicating whether or not cross-component prediction (CCP(Cross-Component Prediction), also referred to as CC prediction) can beused. For example, the cross-component prediction can be used in a casewhere the flag information is “1” (true), and the cross-componentprediction cannot be used in a case where the flag information is “0”(false).

Note that the CCP will also be referred to as cross-component linearprediction (CCLM or CCLMP).

<Prediction Mode Information Pinfo>

The prediction mode information Pinfo includes, for example,information, such as size information PBSize (prediction block size) ofprocessing target PB (prediction block), intra prediction modeinformation IPinfo, and motion prediction information MVinfo.

The intra prediction mode information IPinfo includes, for example,prev_intra_luma_pred_flag, mpm_idx, and rem_intra_pred_mode inJCTVC-W1005, 7.3.8.5 Coding Unit syntax, a luminance intra predictionmode IntraPredModeY derived from the syntax, and the like.

In addition, the intra prediction mode information IPinfo includes, forexample, a cross-component prediction flag (ccp_flag (cclmp_flag)), amulti-class linear prediction mode flag (mclm_flag), a chroma samplelocation type identifier (chroma_sample_loc_type_idx), a chroma MPMidentifier (chroma_mpm_idx), a luminance intra prediction mode(IntraPredModeC) derived from the syntax, and the like.

The cross-component prediction flag (ccp_flag (cclmp_flag)) is flaginformation indicating whether or not to apply cross-component linearprediction. For example, the cross-component prediction is to be appliedin a case of ccp_flag==1, and the cross-component prediction is not tobe applied in a case of ccp_flag==0.

The multi-class linear prediction mode flag (mclm_flag) is informationregarding the mode of linear prediction (linear prediction modeinformation). More specifically, the multi-class linear prediction modeflag (mclm_flag) is flag information indicating whether or not to setthe multi-class linear prediction mode. For example, a 1-class mode(single class mode) (for example, CCLMP) is set in a case of “0,” and a2-class mode (multi-class mode) (for example, MCLMP) is set in a case of“1.”

The chroma sample location type identifier (chroma_sample_loc_type_idx)is an identifier for identifying the type of the pixel location ofchroma component (also referred to as a chroma sample location type).For example, in a case where the chroma array type (ChromaArrayType)that is information regarding the color format indicates 420-format, thechroma sample location type identifiers are allocated as in thefollowing Equation (6).

[Math. 6]

chroma_sample_loc_type_idx==0:Type2

chroma_sample_loc_type_idx==1:Type3

chroma_sample_loc_type_idx==2:Type0

chroma_sample_loc_type_idx==3:Type1  (6)

Note that the chroma sample location type identifier(chroma_sample_loc_type_idx) is transmitted as (stored in) informationregarding pixel location of chroma component (chroma_sample_loc_info()).

The chroma MPM identifier (chroma_mpm_idx) is an identifier indicatingwhich prediction mode candidate in a chroma intra prediction modecandidate list (intraPredModeCandListC) is to be designated as a chromaintra prediction mode.

The motion prediction information MVinfo includes, for example,information, such as merge_idx, merge_flag, inter_pred_idc, ref_idx_LX,mvp_lX_flag, x={0,1}, and mvd (for example, see JCTVC-W1005, 7.3.8.6Prediction Unit Syntax).

Obviously, the information included in the prediction mode informationPinfo is optional, and information other than the abovementionedinformation may be included.

<Transform Information Tinfo>

The transform information Tinfo includes, for example, the followinginformation. Obviously, the information included in the transforminformation Tinfo is optional, and information other than theinformation described below may be included.

Width size TBWSize and height TBHSize of the processing target transformblock (or may be logarithms log 2TBWSize and log 2TBHSize of TBWSize andTBHSize with 2 as a base). Transform skip flag (ts_flag): a flagindicating whether or not to skip the (inverse) primary transform andthe (inverse) secondary transform.

Scan identifier (scanIdx)

Quantization parameter (qp)

Quantization matrix (scaling_matrix (for example, JCTVC-W1005, 7.3.4Scaling list data syntax))

<Residual Information Rinfo>

The residual information Rinfo (for example, see 7.3.8.11 ResidualCoding syntax of JCTVC-W1005) includes, for example, the followingsyntax.

cbf (coded_block_flag): residual data presence/absence flag,

last_sig_coeff_x_pos: last non-zero coefficient X coordinate,

last_sig_coeff_y_pos: last non-zero coefficient Y coordinate,

coded_sub_block_flag: sub-block non-zero coefficient presence/absenceflag,

sig_coeff_flag: non-zero coefficient presence/absence flag,

gr1_flag: flag indicating whether the level of non-zero coefficient isgreater than 1 (also referred to as GR1 flag),

gr2_flag: flag indicating whether the level of non-zero coefficient isgreater than 2 (also referred to as GR2 flag),

sign_flag: sign indicating positive or negative of non-zero coefficient(also referred to as sign code),

coeff_abs_level_remaining: remaining level of non-zero coefficient (alsoreferred to as non-zero coefficient remaining level),

and the like.

Obviously, the information included in the residual information Rinfo isoptional, and information other than the abovementioned information maybe included.

<Filter Information Finfo>

The filter information Finfo includes, for example, the followingcontrol information regarding each filter process.

Control information regarding deblocking filter (DBF),

control information regarding sample adaptive offset filter (SAO),

control information regarding adaptive loop filter (ALF), and

control information regarding other linear/nonlinear filters.

More specifically, the filter information Finfo includes, for example, apicture to be applied with each filter, information designating a regionin the picture, filter On/Off control information of CU unit, filterOn/Off control information regarding boundary of slice and tile, and thelike. Obviously, the information included in the filter informationFinfo is optional, and information other than the abovementionedinformation may be included.

The decoding unit 612 will be further described. The decoding unit 612refers to the residual information Rinfo to derive the quantizationtransform coefficient level qcoef of each coefficient location in eachtransform block. The decoding unit 612 supplies the quantizationtransform coefficient level qcoef to the inverse quantization unit 613.

In addition, the decoding unit 612 supplies the parsed headerinformation Hinfo, prediction mode information Pinfo, quantizationtransform coefficient level qcoef, transform information Tinfo, andfilter information Finfo to each block. Specifically, the information issupplied as follows.

The header information Hinfo is supplied to the inverse quantizationunit 613, the inverse orthogonal transform unit 614, the prediction unit619, and the in-loop filter unit 616. The prediction mode informationPinfo is supplied to the inverse quantization unit 613 and theprediction unit 619. The transform information Tinfo is supplied to theinverse quantization unit 613 and the inverse orthogonal transform unit614. The filter information Finfo is supplied to the in-loop filter unit616.

Obviously, the example described above is just an example, and theexample is not limited to this. For example, each encoding parameter maybe supplied to any processing unit. In addition, other information maybe supplied to any processing unit.

<Inverse Quantization Unit>

The inverse quantization unit 613 executes a process regarding inversequantization. For example, the inverse quantization unit 613 receivesthe transform information Tinfo and the quantization transformcoefficient level qcoef supplied from the decoding unit 612 and scales(inverse quantization) the value of the quantization transformcoefficient level qcoef, based on the transform information Tinfo, toderive the transform coefficient coefI after inverse quantization.

Note that the inverse quantization is performed as an opposite processof the quantization by the quantization unit 514. In addition, theinverse quantization is a process similar to the inverse quantization bythe inverse quantization unit 517. That is, the inverse quantizationunit 517 executes a process (inverse quantization) similar to that ofthe inverse quantization unit 613.

The inverse quantization unit 613 supplies the derived transformcoefficient coefI to the inverse orthogonal transform unit 614.

<Inverse Orthogonal Transform Unit>

The inverse orthogonal transform unit 614 executes a process regardinginverse orthogonal transform. For example, the inverse orthogonaltransform unit 614 receives the transform coefficient coefI suppliedfrom the inverse quantization unit 613 and the transform informationTinfo supplied from the decoding unit 612. The inverse orthogonaltransform unit 614 applies the inverse orthogonal transform process tothe transform coefficient coefI based on the transform information Tinfoto derive the predicted residual resiI.

Note that the inverse orthogonal transform is executed as an oppositeprocess of the orthogonal transform performed by the orthogonaltransform unit 513. In addition, the inverse orthogonal transform is aprocess similar to the inverse orthogonal transform performed by theinverse orthogonal transform unit 518. That is, the inverse orthogonaltransform unit 518 executes a process (inverse orthogonal transform)similar to that of the inverse orthogonal transform unit 614.

The inverse orthogonal transform unit 614 supplies the derived predictedresidual resiI to the computation unit 615.

<Computation Unit>

The computation unit 615 executes a process related to addition ofinformation regarding an image. For example, the computation unit 615receives the predicted residual resiI supplied from the inverseorthogonal transform unit 614 and the predicted image P supplied fromthe prediction unit 619. The computation unit 615 adds the predictedresidual resiI and the predicted image P (prediction signal)corresponding to the predicted residual resiI as illustrated in thefollowing Equation (7), to derive the local decoded image Rlocal.

[Math. 7]

Rlocal=resiI+P  (7)

The computation unit 615 supplies the derived local decoded image Rlocalto the in-loop filter unit 616 and the frame memory 618.

<In-Loop Filter Unit>

The in-loop filter unit 616 executes a process related to an in-loopfilter process. For example, the in-loop filter unit 616 receives thelocal decoded image Rlocal supplied from the computation unit 615 andthe filter information Finfo supplied from the decoding unit 612. Notethat the information input to the in-loop filter unit 616 is optional,and information other than the abovementioned information may be input.

The in-loop filter unit 616 appropriately applies a filter process tothe local decoded image Rlocal based on the filter information Finfo.

For example, the in-loop filter unit 616 sequentially applies fourin-loop filters including a bilateral filter, a deblocking filter (DBF(DeBlocking Filter)), an adaptive offset filter (SAO (Sample AdaptiveOffset)), and an adaptive loop filter (ALF (Adaptive Loop Filter)), inthis order. Note that the filters to be applied and the order ofapplying the filters are optional and can appropriately be selected.

The in-loop filter unit 616 executes a filter process corresponding tothe filter process executed on the encoding side (for example, thein-loop filter unit 520 of the image encoding apparatus 500). Obviously,the filter process executed by the in-loop filter unit 616 is optional,and the filter process is not limited to the example. For example, thein-loop filter unit 616 may apply a Wiener filter or the like.

The in-loop filter unit 616 supplies the filtered local decoded imageRlocal to the rearrangement buffer 617 and the frame memory 618.

<Rearrangement Buffer>

The rearrangement buffer 617 receives the local decoded image Rlocalsupplied from the in-loop filter unit 616 and holds (stores) the localdecoded image Rlocal. The rearrangement buffer 617 uses the localdecoded image Rlocal to reconstruct the decoded image R of each pictureunit and holds the decoded image R (stores the decoded image R in thebuffer). The rearrangement buffer 617 rearranges the obtained decodedimages R from the order of decoding to the order of reproduction. Therearrangement buffer 617 outputs the rearranged decoded image R group asmoving image data to the outside of the image decoding apparatus 600.

<Frame Memory>

The frame memory 618 executes a process regarding storage of datarelated to an image. For example, the frame memory 618 receives thelocal decoded image Rlocal supplied from the computation unit 615 toreconstruct the decoded image R of each picture unit. The frame memory618 stores the decoded image R in the buffer of the frame memory 618.

In addition, the frame memory 618 receives the in-loop-filtered localdecoded image Rlocal supplied from the in-loop filter unit 616 toreconstruct the decoded image R of each picture unit and stores thedecoded image R in the buffer of the frame memory 618. The frame memory618 appropriately supplies the stored decoded image R (or part of thedecoded image R) as a reference image to the prediction unit 619.

Note that the frame memory 618 may store the header information Hinfo,the prediction mode information Pinfo, the transform information Tinfo,the filter information Finfo, and the like regarding the generation ofthe decoded image.

<Prediction Unit>

The prediction unit 619 executes a process regarding generation of apredicted image. For example, the prediction unit 619 receives theprediction mode information Pinfo supplied from the decoding unit 612.The prediction unit 619 uses a prediction method designated by theprediction mode information Pinfo to make a prediction and derives thepredicted image P. In the derivation, the prediction unit 619 uses, as areference image, the decoded image R (or part of the decoded image R)before filtering or after filtering that is designated by the predictionmode information Pinfo and that is stored in frame memory 618. Theprediction unit 619 supplies the derived predicted image P to thecomputation unit 615.

<Application of Present Technique>

In the image decoding apparatus 600 configured in such a way, thedecoding apparatus 180 (FIG. 19) may be applied as the decoding unit612, for example. That is, the decoding unit 612 may include the scancontrol unit 181 and the decoding unit 182, and various types of scancontrol described in <3. Concept> to <6. Third Embodiment> may beapplied to the decoding performed by the decoding unit 182.

According to the configuration, the decoding unit 612 can correctlydecode the encoded data generated in the order of scan for reducing theunnecessary information included in the bitstream (the encoded data ofthe coefficient data of the sub-blocks in the invalid transformcoefficient region) as in the case of the decoding apparatus 180. Thatis, the decoding unit 612 can suppress the reduction in the encodingefficiency. In addition, decoding of the unnecessary information can beskipped, and therefore, the increase in the load of the decoding processcan be suppressed. The increase in the cost, the circuit scale, theprocessing time, and the like can thus be suppressed.

Therefore, the image decoding apparatus 600 can suppress the reductionin the encoding efficiency. In addition, the image decoding apparatus600 can suppress the increase in the load of the decoding process andsuppress the increase in the cost, the circuit scale, the processingtime, and the like.

<Flow of Image Decoding Process>

Next, an example of a flow of an image decoding process executed by theimage decoding apparatus 600 will be described with reference to a flowchart of FIG. 46.

Once the image decoding process is started, the accumulation buffer 611acquires the encoded data (bitstream) supplied from the outside of theimage decoding apparatus 600 and holds (accumulates) the encoded data instep S401.

In step S602, the decoding unit 612 decodes the encoded data (bitstream)and obtains the quantization transform coefficient level qcoef. Further,in the decoding, the decoding unit 612 parses (analyzes and acquires)various encoding parameters from the encoded data (bitstream).

In step S403, the inverse quantization unit 613 applies, to thequantization transform coefficient level qcoef obtained in the processof step S402, inverse quantization that is an opposite process of thequantization performed on the encoding side and obtains the transformcoefficient coefI.

In step S404, the inverse orthogonal transform unit 614 applies, to thetransform coefficient coefI obtained in the process of step S403, aninverse orthogonal transform process that is an opposite process of theorthogonal transform process performed on the encoding side and obtainsthe predicted residual resiI.

In step S405, the prediction unit 619 uses a prediction methoddesignated on the encoding side, to execute the prediction process basedon the information parsed in step S402, and generates the predictedimage P by, for example, referencing the reference image stored in theframe memory 618.

In step S406, the computation unit 615 adds the predicted residual resiIobtained in the process of step S404 and the predicted image P obtainedin the process of step S405, to derive the local decoded image Rlocal.

In step S407, the in-loop filter unit 616 applies the in-loop filterprocess to the local decoded image Rlocal obtained in the process ofstep S406.

Further, in step S408, the frame memory 618 stores at least one of thelocal decoded image Rlocal obtained in the process of step S406 or thelocal decoded image Rlocal after filter process obtained in the processof step S407.

In step S409, the rearrangement buffer 617 uses the filtered localdecoded image Rlocal obtained in the process of step S407, to derive thedecoded image R, and rearranges the order of the decoded image R groupfrom the order of decoding to the order of reproduction.

In step S410, the rearrangement buffer 617 outputs, as moving images,the decoded image R group rearranged in the order of reproduction to theoutside of the image decoding apparatus 600. Once the process of stepS410 is completed, the image decoding process ends.

In the decoding process executed in step S402 of the image decodingprocess, the order of scan of the valid transform coefficient region iscontrolled as described above. That is, the decoding process asdescribed with reference to the flow chart of FIG. 20 may be executed inthis decoding process. In such a way, the image decoding apparatus 600can further suppress the reduction in the encoding efficiency.

<7-3: Syntax>

In the case of applying the present technique to the image encodingapparatus 500 or the image decoding apparatus 600, although the controlof the scan of the present technique may be realized by changing theorder of scan as described above, the syntax may be used to realize thecontrol.

FIG. 47 illustrates an example of the syntax of the transform unit(transform_unit). As indicated in the tenth to sixteenth stages from thetop of FIG. 47, the residual data is encoded (residual_coding) for theluminance components (luma) and the chroma components (cb, cr).

FIG. 48 illustrates an example of the syntax for encoding the residualdata (coefficient data). In the syntax, the encoding of the coefficientdata is basically controlled similarly to the example of FIG. 6.Further, as in the example of FIG. 21, the range of the scan is limitedto the valid transform coefficient region ((xS<=32) && (yS<=32)) in theif statement of the thirty-second stage from the top. In such a way, thescan control can easily be realized.

8. Note <Computer>

The series of processes can be executed by hardware or can be executedby software. In the case where the series of processes is executed bysoftware, a program included in the software is installed on a computer.Here, examples of the computer include a computer incorporated intodedicated hardware and a general-purpose personal computer that canexecute various functions by installing various programs.

FIG. 49 is a block diagram illustrating a configuration example of thehardware of the computer that uses a program to execute the series ofprocesses.

In a computer 800 illustrated in FIG. 49, a CPU (Central ProcessingUnit) 801, a ROM (Read Only Memory) 802, and a RAM (Random AccessMemory) 803 are connected to each other through a bus 804.

An input-output interface 810 is also connected to the bus 804. An inputunit 811, an output unit 812, a storage unit 813, a communication unit814, and a drive 815 are connected to the input-output interface 810.

The input unit 811 includes, for example, a keyboard, a mouse, amicrophone, a touch panel, an input terminal, and the like. The outputunit 812 includes, for example, a display, a speaker, an outputterminal, and the like. The storage unit 813 includes, for example, ahard disk, a RAM disk, a non-volatile memory, and the like. Thecommunication unit 814 includes, for example, a network interface. Thedrive 815 drives a removable medium 821, such as a magnetic disk, anoptical disk, a magneto-optical disk, and a semiconductor memory.

In the computer configured in such a way, the CPU 801 loads, forexample, a program stored in the storage unit 813 to the RAM 803 throughthe input-output interface 810 and the bus 804 to execute the programand thereby execute the series of processes. Data and the like necessaryfor the CPU 801 to execute various processes are also appropriatelystored in the RAM 803.

The program executed by the computer (CPU 801) can be applied by, forexample, recording the program in the removable medium 821 as a packagemedium or the like. In this case, the removable medium 821 can bemounted on the drive 815 to install the program on the storage unit 813through the input-output interface 810.

Further, the program can also be provided through a wired or wirelesstransmission medium, such as a local area network, the Internet, anddigital satellite broadcasting. In this case, the program can bereceived by the communication unit 814 and installed on the storage unit813.

In addition, the program can also be installed in advance on the ROM 802or the storage unit 813.

<Units of Information and Processes>

The data units in setting various types of information and the dataunits in various processes described above are optional, and the dataunits are not limited to the examples. For example, the information andthe processes may be set for each TU (Transform Unit), TB (TransformBlock), PU (Prediction Unit), PB (Prediction Block), CU (Coding unit),LCU (Largest Coding Unit), sub-block, block, tile, slice, picture,sequence, or component, or the data in the data units may be the target.Obviously, the data unit can be set for each piece of information oreach process, and not all of the data units of the information and theprocesses have to be uniform. Note that the storage location of theinformation is optional, and the information may be stored in theheader, the parameter set, or the like of the data unit. In addition,the information may be stored in plural places.

<Control Information>

The control information regarding the present technique described ineach of the embodiments may be transmitted from the encoding side to thedecoding side. For example, control information (for example,enabled_flag) for controlling whether or not to permit (or prohibit) theapplication of the present technique may be transmitted. In addition,for example, control information indicating the target of applying thepresent technique (or the target of not applying the present technique)may be transmitted. For example, control information for designating theblock size (one of or both an upper limit and a lower limit), the frame,the component, the layer, or the like in which the present technique isapplied (or in which the application of the present technique ispermitted or prohibited) may be transmitted.

<Target of Present Technique>

The present technique can be applied to any image encoding and decodingsystems. That is, the specifications of various processes regardingimage encoding and decoding, such as transform (inverse transform),quantization (inverse quantization), encoding (decoding), andprediction, are optional as long as the specifications do not contradictthe present technique, and the specifications are not limited to theexamples. In addition, part of the processes may be skipped unless itcontradicts the present technique.

In addition, the present technique can be applied to a multi-view imageencoding and decoding system that performs encoding and decoding ofmulti-view images including images from plural viewpoints (views). Inthat case, it is sufficient if the present technique is applied to theencoding and decoding from each viewpoint (view).

Further, the present technique can be applied to a tiered image encoding(scalable encoding) and decoding system that encodes and decodes tieredimages divided into plural layers (tiers) to provide a scalabilityfunction for a predetermined parameter. In that case, it is sufficientif the present technique is applied to the encoding and decoding of eachtier (layer).

The image processing apparatus, the image encoding apparatus, and theimage decoding apparatus according to the embodiments can be applied tovarious electronic devices including, for example, a transmitter and areceiver (for example, television receiver and portable phone) indistribution through satellite broadcasting, cable broadcasting such ascable TV, or the Internet or in distribution to a terminal throughcellular communication, an apparatus (for example, hard disk recorderand camera) that records images in a medium, such as an optical disk, amagnetic disk, and a flash memory, and that reproduces images from thesestorage media, and the like.

In addition, the present technique can also be carried out in anyconfiguration mounted on any apparatus or an apparatus included in asystem, such as, for example, a processor (for example, video processor)as system LSI (Large Scale Integration) or the like, a module (forexample, video module) using plural processors or the like, a unit (forexample, video unit) using plural modules or the like, and a set (forexample, video set) provided with other functions in addition to theunit (that is, configuration of part of an apparatus).

Further, the present technique can also be applied to a network systemincluding plural apparatuses. For example, the present technique can beapplied to a cloud service for providing a service regarding images(moving images) to any terminal, such as a computer, an AV (AudioVisual) device, a portable information processing terminal, and an IoT(Internet of Things) device.

Note that the systems, the apparatuses, the processing units, and thelike according to the present technique can be used in any fields, suchas, for example, traffic, medical care, crime prevention, agriculture,livestock industry, mining industry, cosmetics, factories, homeappliances, weather, and natural surveillance. In addition, the usage isalso optional.

For example, the present technique can be applied to a system or adevice used for providing content to be viewed or the like. In addition,the present technique can also be applied to, for example, a system or adevice used for traffic, such as for monitoring traffic conditions andfor controlling automatic driving. Further, the present technique canalso be applied to, for example, a system or a device used for security.In addition, the present technique can be applied to, for example, asystem or a device used for automatic control of a machine or the like.Further, the present technique can also be applied to, for example, asystem or a device used for agriculture or livestock industry. Inaddition, the present technique can also be applied to, for example, asystem or a device that monitors the state of nature, such as volcanos,forests, and oceans, wild life, and the like. Furthermore, the presenttechnique can also be applied to, for example, a system or a device usedfor sports.

<Others>

Note that the “flag” in the present specification is information foridentifying plural states, and the “flag” includes not only informationused for identifying two states of true (1) and false (0), but alsoinformation that can identify three or more states. Therefore, thenumber of possible values of the “flag” may be two, such as 1/0, or maybe three or more. That is, the number of bits of the “flag” is optional,and the “flag” may include 1 bit or plural bits. In addition, as for theidentification information (including flag), there can be not only aform of including the identification information in the bitstream, butalso a form of including difference information of the identificationinformation with respect to reference information in the bitstream.Therefore, the “flag” and the “identification information” in thepresent specification include not only the information of the “flag” andthe “identification information,” but also the difference informationwith respect to the reference information.

In addition, various types of information (metadata and the like)regarding encoded data (bitstream) may be transmitted or recorded in anyform as long as the information is associated with the encoded data.Here, the term “associated” denotes, for example, that one piece of datacan be used (can be linked) in processing another piece of data. Thatis, the pieces of data associated with each other may be integrated asone piece of data or may be provided as separate pieces of data. Forexample, the information associated with the encoded data (image) may betransmitted on a transmission path different from the encoded data(image). In addition, for example, the information associated with theencoded data (image) may be recorded in a recording medium separate fromthe encoded data (image) (or in a separate recording area of the samerecording medium). Note that part of the data may be “associated,”instead of the entire data. For example, the image and the informationcorresponding to the image may be associated with each other in anyunit, such as plural frames, one frame, and part of the frame.

Note that the terms, such as “combine,” “multiplex,” “add,” “integrate,”“include,” “store,” “put in,” “place into,” and “insert,” in the presentspecification denote grouping of plural things into one thing, such asgrouping of encoded data and metadata into one piece of data, and eachterm denotes one method of “associating” described above.

In addition, the embodiments of the present technique are not limited tothe embodiments described above, and various changes can be made withoutdeparting from the scope of the present technique.

In addition, for example, the configuration of one apparatus (orprocessing unit) described above may be divided to provide pluralapparatuses (or processing units). Conversely, the configurations ofplural apparatuses (or processing units) described above may beintegrated to provide one apparatus (or processing unit). In addition,configurations other than the configurations described above mayobviously be added to the configuration of each apparatus (or eachprocessing unit). Further, part of the configuration of an apparatus (orprocessing unit) may be included in the configuration of anotherapparatus (or another processing unit) as long as the configuration andthe operation of the entire system are substantially the same.

Note that the system in the present specification denotes a set ofplural constituent elements (apparatuses, modules (components), and thelike), and whether or not all of the constituent elements are in thesame housing does not matter. Therefore, plural apparatuses stored inseparate housings and connected through a network and one apparatusstoring plural modules in one housing are both systems.

In addition, the present technique can be provided as, for example,cloud computing in which plural apparatuses share one function andcooperate to execute a process through a network.

In addition, the program described above can be executed by, forexample, any apparatus. In that case, it is sufficient if the apparatuscan have necessary functions (such as functional blocks) and obtainnecessary information.

In addition, for example, one apparatus can execute each step describedin the flow charts, or plural apparatuses can take charge and executeeach step. Further, in the case where one step includes pluralprocesses, one apparatus can execute the plural processes included inone step, or plural apparatuses can take charge and execute theprocesses. In other words, plural processes included in one step can beexecuted as processes of plural steps. Conversely, processes describedas plural steps can be integrated and executed in one step.

Note that the program executed by the computer may be a program in whichthe processes of the steps describing the program are executed inchronological order described in the present specification, or theprogram may be a program for executing the processes in parallel or forexecuting the processes separately at a necessary timing such as whenthe processes are invoked. That is, the processes of the steps may beexecuted in an order different from the order described above as long asthere is no contradiction. Further, the processes of the stepsdescribing the program may be executed in parallel with processes ofother programs or may be executed in combination with processes of otherprograms.

Note that the plural present techniques described in the presentspecification can independently and separately be carried out as long asthere is no contradiction. Obviously, any plural present techniques canalso be combined and carried out. For example, part or all of thepresent technique described in one of the embodiments can also becarried out in combination with part or all of the present techniquedescribed in another embodiment. In addition, part or all of any presenttechnique described above can also be carried out in combination withanother technique not described above.

REFERENCE SIGNS LIST

-   -   170: Encoding apparatus    -   171: Scan control unit    -   172: Encoding unit    -   180: Decoding apparatus    -   181: Scan control unit    -   182: Decoding unit    -   500: Image encoding apparatus    -   501: Control unit    -   515: Encoding unit    -   600: Image decoding apparatus    -   612: Decoding unit

1. An image processing apparatus comprising: an encoding unit that skipsencoding of coefficient data related to an image in an invalid transformcoefficient region and that encodes the coefficient data in a validtransform coefficient region.
 2. The image processing apparatusaccording to claim 1, wherein the coefficient data includes validtransform coefficient flag information indicating presence/absence of avalid transform coefficient for each sub-block, and the encoding unitskips encoding of the valid transform coefficient flag information inthe invalid transform coefficient region and encodes the valid transformcoefficient flag information in the valid transform coefficient region.3. The image processing apparatus according to claim 2, wherein thecoefficient data further includes validity flag information indicating,for each sub-block, presence/absence of a valid transform coefficientwith a value that is not 0 and includes a sign and an absolute value ofeach valid transform coefficient, and the encoding unit encodes thevalidity flag information of the sub-block in which the valid transformcoefficient flag information is encoded and encodes the sign and theabsolute value of each valid transform coefficient of the sub-block inwhich the validity flag information is true.
 4. The image processingapparatus according to claim 1, wherein the encoding unit skips encodingof the coefficient data in the invalid transform coefficient region andencodes the coefficient data in the valid transform coefficient region,based on information indicating a block size of a block to be processedand information indicating a range of the valid transform coefficientregion.
 5. The image processing apparatus according to claim 1, whereinthe encoding unit encodes the coefficient data in the valid transformcoefficient region in a scan order corresponding to the coefficientdata.
 6. The image processing apparatus according to claim 5, whereinthe encoding unit scans and encodes the coefficient data in the validtransform coefficient region in a vertical direction or a horizontaldirection for each sub-block.
 7. The image processing apparatusaccording to claim 5, wherein the encoding unit encodes, for eachsub-block, the coefficient data in the valid transform coefficientregion in a scan order corresponding to a shape of the block to beprocessed.
 8. The image processing apparatus according to claim 5,wherein the encoding unit scans and encodes the coefficient data in thevalid transform coefficient region for each group of sub-blocks.
 9. Theimage processing apparatus according to claim 1, wherein, for luminancecomponents and chroma components of the image, the encoding unit skipsencoding of the coefficient data in the invalid transform coefficientregion and encodes the coefficient data in the valid transformcoefficient region.
 10. An image processing method comprising: skippingencoding of coefficient data related to an image in an invalid transformcoefficient region and encoding the coefficient data in a validtransform coefficient region.
 11. An image processing apparatuscomprising: a decoding unit that skips, in an invalid transformcoefficient region, decoding of encoded data including encodedcoefficient data related to an image and that decodes the encoded datain a valid transform coefficient region.
 12. The image processingapparatus according to claim 11, wherein the coefficient data includesvalid transform coefficient flag information indicating presence/absenceof a valid transform coefficient for each sub-block, and the decodingunit decodes encoded data of the valid transform coefficient flaginformation in the valid transform coefficient region.
 13. The imageprocessing apparatus according to claim 12, wherein the coefficient datafurther includes validity flag information indicating, for eachsub-block, presence/absence of a valid transform coefficient with avalue that is not 0 and includes a sign and an absolute value of eachvalid transform coefficient, and the decoding unit decodes encoded dataof the validity flag information of the sub-block for which the encodeddata of the valid transform coefficient flag information is decoded anddecodes encoded data of the sign and the absolute value of each validtransform coefficient of the sub-block in which the validity flaginformation is true.
 14. The image processing apparatus according toclaim 11, wherein the decoding unit skips decoding of the encoded datain the invalid transform coefficient region and decodes the encoded datain the valid transform coefficient region, based on informationindicating a block size of a block to be processed and informationindicating a range of the valid transform coefficient region.
 15. Theimage processing apparatus according to claim 11, wherein the decodingunit decodes the encoded data in the valid transform coefficient regionin a scan order corresponding to the coefficient data.
 16. The imageprocessing apparatus according to claim 15, wherein the decoding unitscans and decodes the encoded data in the valid transform coefficientregion in a vertical direction or a horizontal direction for eachsub-block.
 17. The image processing apparatus according to claim 15,wherein the decoding unit decodes, for each sub-block, the encoded datain the valid transform coefficient region in a scan order correspondingto a shape of the block to be processed.
 18. The image processingapparatus according to claim 15, wherein the decoding unit scans anddecodes the encoded data in the valid transform coefficient region foreach group of sub-blocks.
 19. The image processing apparatus accordingto claim 11, wherein, for luminance components and chroma components ofthe image, the decoding unit skips decoding of the encoded data in theinvalid transform coefficient region and decodes the encoded data in thevalid transform coefficient region.
 20. An image processing methodcomprising: skipping decoding of encoded data including encodedcoefficient data related to an image in an invalid transform coefficientregion and decoding the encoded data in a valid transform coefficientregion.